A New Zerbet’s Statistic for Detecting Outliers in Exponential Distribution

Introduction

The problem of detecting outliers is universal. In the presence of good theoretical tools, an analysis based on erroneous data often leads to false conclusions, hence the importance of developing statistical tests to detect anomalies and ensure the quality of the data is warranted. A so-called outlier value does not imply that this value is necessarily not useful and interesting, but it presents the extreme case of a particularly important phenomenon. This explains why we do not seek to remove outliers but to minimize their undesirable effects when using them. The objective of this paper is to focus on alternative models, namely slippage alternatives in exponential distribution [1]. This distribution has several applications in various fields, such as queueing systems, reliability engineering, survival analysis, etc. It is specifically used to model processes or events that occur with a constant rate (or intensity) λ. The rate λ determines the average number of events or occurrences per unit of time. An exponential random variable can be regarded as the time between consecutive events in a Poisson process with a constant rate λ. For this distribution, a number of discordancy tests for multiple upper outliers have been proposed by various authors; for example, see [2-7].

The Proposed New Test Statistic

In this paper, to test H0 against Hr, we propose the new following statistic:

The Probability Density Function of The New Statistic

The main objective in this section is to obtain the probability density function of the statistic Z(r, n), with r ≥ 1 in the explicit form, under the slippage alternative Hr and also under null hypothesis.

Critical values

The main objective in this section is to give the critical values of the six tests ZN, D₁, D₂, BG, MLR and Z computed by simulation using the Monte Carlo method (100 000 replications) at the 0.2,0.15,0.1,0.05,0.02 and 0.01 levels of significance, for r = 2, n = 20, 40,...,100, the results are summarized in the tables 1, 2, ..., 6 below.

According to the above tables, we can see the critical values of 1, D₂, BG and MLR decrease when n is increased. But the critical values of ZN and Z increase when n is increased.

Simulations Experiments

Under the null hypothesis, the distribution of any of the considered test statistic is parameterfree; hence, investigating by simulation the power of the tests, the value σ = 1 could be taken when simulating the data under the null hypothesis, and different values of b could be considered for fixed σ = 1 when simulating the data under the alternative hypothesis H_r.

Assume that r = 2. For performance comparison of the six tests ZN, D_1, D₂, BG, MLR and Z, we generated N = 100 000 samples of size n = 20 from the alternative distributions considered Hr. We computed then the power of the six tests by simulation for various values of b. The results are presented in Table 7. A good test should have a high power.

Table 7: Empirical Power for n=20, r=2 and α = 0.05

From Table 7, we observed that:

The powers of the six tests ZN, D1, D2, BG, MLR and Z increase to 1 when b increases;

The D1 and D2 tests are equivalent; For b ≥ 3, the ZN test has more power than the D₁ and D₂ tests;

The test based on the BG test is more powerful than the D₁, D₂ and ZN tests;

The power of MLR test increases to 1 rapidly than the D₁, D₂, ZN and BG tests;

The power of the new Z test increases to 1 very rapidly than the other tests D₁, D₂, BG, MLR and ZN. The curves of the empirical power of different test statistics at n = 20, r = 2 and α = 0.05 are depicted in the following figure:

                           Figure 1: Empirical Power of Tests

Conclusion

The power analysis shows that, for exponential distribution, the Dixon₁, Dixon₂, Zerbet and Nikulin, and Balasooriya and Gadag tests have considerably smaller power than the new Zerbet and Maximum Likelihood Ratio tests. The power of the new Zerbet test increases to 1 very rapidly than the Maximum Likelihood Ratio test. We conclude that, for exponential distribution, the new Zerbet test has a higher probability of identifying the contaminant observations as outliers than the Dixon₁, Dixon_2, Zerbet and Nikulin, Balasooriya and Gadag, and Maximum Likelihood Ratio tests for all values of b > 1 [2-8].

Acknowledgement

I would like to express my sincere gratitude the anonymous referees, an Associate Editors and the Editors for their constructive comments that improved the quality of this paper.

Acknowledgement

I would like to express my sincere gratitude the anonymous referees, an Associate Editors and the Editors for their constructive comments that improved the quality of this paper.

References

1. Barnett, V. and Lewis, T. (1978). Outlier in statistical data. John Wiley and Sons, Inc: New York.
2. Likes, J. (1967). Distribution of Dixon’s statistics in the case of an exponential population. Metrika, 11(1), 46-54.
3. Chikkagoudar, M. S., & Kunchur, S. H. (1983). Distributions of test statistics for multiple outliers in exponential samples.Communications in Statistics-theory and Methods, 12(18), 2127-2142.
4. Likes, J. (1987). Some Tests for k£ 2 Upper Outliers in an Exponential Sample. Biometrical Journal, 29(3), 313-321.
5. Balasooriya, U., & Gadag, V. (1994). Tests for upper outliers in the two-parameter exponential distribution. Journal of statistical computation and simulation, 50(3-4), 249-259.
6. Zhang, J. (1998). Tests for multiple upper or lower outliers in an exponential sample. Journal of Applied Statistics, 25(2), 245-255.
7. Zerbet, A., & Nikulin, M. (2003). A new statistic for detecting outliers in exponential case. Communications in Statistics-theory and Methods, 32(3), 573-583.
8. Lin, C. T., & Balakrishnan, N. (2014). Tests for multiple outliers in an exponential sample. Communications in Statistics-Simulation and Computation, 43(4), 706-722.
9. Bagdonavičius, V., Nikulin, M. S., & Zerbet, A. (2015). On outliers detection for location-scale and shape-scale families. Notes ofscientific seminars of POMI, 442(0), 5-17.
10. Bagdonavicius, V., Nikulin, M. and Zerbet, A. (2017). On Outliers Detection for Location-Scale and Shape-Scale Families. Journal of Mathematical Sciences, 225, 723-732.
11. Pyke, R. (1965). Spacings. Journal of Royal Statistical Society, 3, pp. 395-449.
12. Sweeting, T. J. (1986). Asymptotically independent scale-free spacings with applications to discordancy testing. The Annals of Statistics, 1485-1496.