METHOD OF TESTING THE READINESS OF MEANS OF TRANSPORT WITH THE USE OF SEMI-MARKOV PROCESSES

In the analysis of the readiness of means of transport, the Markov and semi-Markov processes are particularly applicable, allowing for the description of the usage process over long periods of time, determination of indicators of the exploitability and readiness of the used set of objects, as well as simulation of long-term forecasts of the usage process results. The studies presented in the literature usually concern the theoretical side of the matter, mainly the construction of formal models of the process of changing the operating states of a vehicle. Less attention is paid to the empirical side, especially with regard to the actual conditions of use. Examples of experimental observations presented in the literature most often concern individual cases. This paper lists selected irregularities and presents an example of a study of a real transport system based on semi-Markov processes.


Introduction
Markov and semi-Markov models are used, when the analysed states (failures, deterioration, service) occur at random moments (Buchholz et al. 2018). Thus, their diagnostic and prognostic evaluation is possible. Diagnostics is used to determine the current state of the analysed system/element, while prognoses are used to determine its limit indications, like the remaining service life.
There are many such studies available in the literature, especially in terms of technical facilities readiness and reliability evaluation. In transport engineering or logistics (Darong et al. 2018;Wang, Infield 2018), the theory of one-dimensional Markov and semi-Markov processes is mainly used to describe individual means of transport, treated as a set of functional elements, e.g., a passenger car (Girtler 2012), a bus (Landowski et al. 2017) or a helicopter (Woropay et al. 2004). Considerations concern the determination of their service times (Buchholz et al. 2018;Bunks et al. 2000) or possible scenarios of the aging process (Limnios, Oprişan 2001). Selected components are also examined, such as a compression ignition engine in paper by Rudnicki (2011) or a motor drive system of urban transit in paper by Darong et al. (2018).
Comprehensive analyses allow the formulation and optimization of maintenance policy, as Chen and Trivedi (2005) or Lisnianski and Frenkel (2009) presented in their papers. The systems are analysed as a whole (Borucka et al. 2019;Knopik, Migawa 2017), or the individual phases are considered independently, and each of them is described by a separate Markov model (Alam, Al-Saggaf 1986). Studies describing complex, actual structures are not popular, and those that deserve attention include (Love et al. 2000;Restel 2014;Woropay et al. 2004;Żurek, Tomaszewska 2016). This indicates the need to present methods for modelling such systems using Markov processes, which is also addressed by this paper.
The review of available literature has shown that not all authors using Markov processes bear in mind the assumptions for their application. This is noted for example in papers by Kozłowski et al. (2020), Shi et al. (2013), Zhang et al. (2010). The first requirement is to meet the Markov property. The prediction can be precise only when random sequences of variables meet the Markov property Kozłowski et al. 2020). It is therefore necessary to test the randomness of the sequence of statistical data collected, as was made in papers by  and Sanusi et al. (2015). The second assumption concerns the distribution of the variable studied. Many authors point out that this allows to obtain reliable results (Li et al. 2010;Perman et al. 1997). Goodness of fit of empirical data to the parametric distribution is presented, among others, in papers by Van Casteren et al. (2000) -Weibull distribution, Li et al. (2010) -Poisson distribution, Rydén et al. (1998) -double exponential distribution. It happens, though, that only the assumption concerning the form of distributions is made, while the choice of the model is justified by the accuracy of the results obtained. This does not, however, permit the use of models without satisfying the sample requirements. The obtained forecasts may be correct for a dozen or so test observations, but this does not confirm the validity of using the model for the boundary indications of the process, for n → ∞.
An analysis of distributions of variables studied and an assessment of the possibility of fitting them to a known parametric family determines the possibility of using a specific model. It is the Markov model in the case of exponential distributions, and semi-Markov model in the case of other than exponential ones. This paper presents the way of estimating semi-Markov model parameters on the basis of actual observations obtained from the transport company, which additionally allowed for diagnostics and evaluation of the company in terms of its readiness.

Material and methods: introduction to the Markov processes
The literature presents different definitions of semi-Markov processes (Knopik, Migawa 2017;Tang et al. 2018;Zhang et al. 2012). On the basis of works by Grabski, Jaźwiński (2009) and Grabski (2015), it was assumed to define the semi-Markov process with a finite set of states by means of the so-called Markov renewal process.
Let N and N 0 denote a set of natural numbers and a set of non-negative integers. In addition, let S denote a finite or countable set, while R + a set of non-negative real numbers.

Let ( )
, , Ω Ρ  denote the probabilistic space in which a sequence of two-dimensional random variables ( ) { } 0 , : n n n N ξ ϑ ∈ , is defined, such that : n S ξ Ω → and : Definition 1 (Papamichail et al. 2016;Hunter 2016;Asmussen et al. 2016 with a probability of 1; 2) for all , i j S ∈ : functional matrix: where: is called the renewal kernel. Vector : is an initial distribution of Markov renewal process. The renewal kernel meets the conditions: are non-decreasing and right-sided continuous functions; The semi-Markov process with a discrete set of states S is defined as follows: then the random process is called a counting process.

Definition 2. The stochastic process
Is called the semi-Markov process generated by the Markov renewal process ( ) { } 0 , : n n n N ξ ϑ ∈ with the initial decomposition p and kernel ( ) Thus defined semi-Markov process is a stochastic one with a discrete space of states S at time t T R + ∈ = . The semi-Markov process is defined when its kernel and initial distribution are specified. The definition shows that: This means that functions that take fixed values in the ranges constitute realizations of the semi-Markov process.
From the definition of both the Markov renewal and SM process, it follows that the state of the semi-Markov process and its duration depends only on the previous state, and not on previous states and their duration. The process kernel and the initial distribution fully define the semi-Markov process (Duan et al. 2019;Wu et al. 2019).

Characteristics of the study subject
The subject of the study is a large enterprise providing road transportation services in Poland. Its fleet consists of 60 Volvo FH 500 trucks (model year: 2012) equipped with compression-ignition engines with a capacity of 13000 cc and with a power of 500 hp. They operate mainly on the basis of long-term agreements with several large production plants, which results in high repeatability of transportation tasks. The company uses modern fleet management software, which archives data on all activities performed by its vehicles. The latter became the basis for the conducted research. The records made available were readings from the vehicles' on-board computers and were sorted and processed from 60 multi-line sheets. After these operations, seven operating states were identified and analysed in detail.
The following operating states were distinguished: »» state S1 -travel, meaning the physical implementation of the transport process; »» state S2 -readiness of vehicle with driver, meaning that the vehicle is operational, in working order and prepared to perform the task. Additionally, the driver is present at the vehicle, awaiting instructions, persons or documents necessary to continue the transportation task, and has met regulatory rest requirements before driving the vehicle; »» state S3 -readiness of vehicle at the stand, meaning that the vehicle is operational, in working order and prepared to perform the task and the driver is absent from the vehicle. This state is usually associated with the vehicle being parked in the garage or in the yard after finishing the shift; »» state S4 -handling works are activities related to vehicle operations on cargo, such as loading, transhipment and unloading; »» state S5 -current maintenance, i.e. inspecting the vehicle before departure and after return, including checking the levels of vehicle fluids, the efficiency of the vehicle lighting, the condition of tires and, if necessary, their replacement; »» state S6 -periodical maintenance, resulting from scheduled service and the vehicle's mileage as well as minor repairs. This includes diagnostics of the technical condition of the vehicle including the following systems: cooling, hydraulic, steering, braking, electric and the powertrain. In addition, in the studied period, the replacements of tie rods, air dryers, brake discs and pads, motor oil, gear fluid in the rear differential, gear fluid in the transmission, oil filter, air filter and cabin pollen filter were recorded; »» state S7 -long-term repairs (in the analysed period these included: head gasket replacement, clutch replacement, suspension system repair, removal and disassembly of the engine with replacement of piston rings). For each distinguished state, basic measures of descriptive statistics were calculated and visual inspection of diagrams was made. The tests were carried out for each vehicle individually as well as collectively for the entire group, concluding that the results obtained for the sample are representative of individual vehicles. With such a large number of observations, drawing conclusions was made possible on the basis of the histograms of probability distribution, as in such a case the histogram takes the shape of the probability density function graph. The scope of the conducted research was presented on the example of the current maintenance operating state.
At the beginning, the graph of the distribution of S5 state duration for the whole sample was analysed, which is shown in Figure 1. This is a naturally limited distribution on the left-hand side with the minimum duration of the state, which in this case is about 5 min. Moreover, it is single-modal with right-sided asymmetry, which is proved by both a positive skew and a higher expected value than the median ( Table 1). The match to several families of parametric distributions was examined. Normal, logarithmically normal, exponential and Weibull distributions were selected. Then, the best model was determined based on the Akaike information criterion, which is a normed match index (Rymarczyk et al. 2019). In this case it turned out to be Weibull distribution with the parameters presented in Table 2.
A confirmation of goodness of fit of empirical data to a selected theoretical distribution was done by comparing the frequencies of real data observation to those of the expected theoretical distribution. In this case, the chisquared test and the Kolmogorov-Smirnov test at the significance level of a = 0.05 were used. The obtained results are presented in Table 3.
Despite the fact that the distribution of the duration of the current maintenance state resembles the Weibull distribution to quite an extent and the distribution diagrams are very similar (Figure 2), the null hypothesis on the compatibility of distributions (p-value = 0.00) was rejected in both the chi-squared and Kolmogorov-Smirnov statistical tests.  Similar conclusions applied to all operating states and the similarity to the Weibull and log-normal distributions, could be observed, this was in no case confirmed by statistical tests. This is illustrated by one of the problems related to the analysis of large data sets. In the case of a large sample, even a small deviation will show that the distribution is not well matched. However, in the case of a small sample, it is more difficult to identify a significant deviation even if it is present .
The next stage of the study was the analysis of basic measures of descriptive statistics. The statistics relevant to the current maintenance state are presented in Table 1.
The current maintenance state lasted nearly 30 min on average, but the median is lower and is over 26 min. The minimum and maximum values are 10 and 49 min respectively, so the range is fairly large and equals 39 min. The coefficient of variation at 38% is satisfactory for such processes.
The basic conclusion of the analyses was to limit the availability of some modelling methods. It was not possible to estimate the Markov model, as the requirement condition for its application is the exponential form of distributions of the analysed variables. Therefore, it was decided to formulate the vehicle exploitation model as a semi-Markov process, which does not impose any requirements concerning the form of distribution so they can be freely concentrated in the set ) 0, ∞ .

Semi-Markov model estimation
As already mentioned, the use of the Markov or semi-Markov processes is subject to certain limitations. The basic requirement for their application is the nature of probability distributions of time spent in states, which for Markov processes must be exponential. In semi-Markov processes the probability distributions can be arbitrary. This extension results in a more complex mathematical apparatus (Gupta, Tyagi 2019;Grabski 2017). The second requirement is the so-called memorylessness of the Markov process, which results directly from the definition. Therefore, before estimating the parameters of the model, it was necessary to verify the above assumptions.
First of all, the said assumption about the memorylessness of the studied process was checked. The relation of a given operating state with the duration and type of previous states was investigated. It began with the Kruskal-Wallis test (H = 4.145, p-value = 0.387). The obtained results of the probability value confirmed the expected independence, which was then validated by testing Markov properties using the R environment (The R Foundation 2019). All possible sequences of events (state transitions) were checked by performing chi-square tests on a number of tables created on the basis of empirical interstate relations. The statistics of chi-square test were 2 1054.6 χ = , while p-value = 0.112, which means that the Markov property is fulfilled. Then, for each state, the basic measures of descriptive statistics were determined and the analysis of time distributions of individual states was made in order to adjust them to the known (parametric) family. Such research enabled the choice of the right mathematical model, determination of its parameters and basic characteristics describing the process and its asymptotic properties.
The first stage of the description of the semi-Markov process is to determine the transition matrix between individual states of the process. On the basis of actual observations, it was determined which transitions are possible and which do not occur. The relations determined in this way are presented in the form of a diagram in the Figure 3.
However, the initial distribution was assumed to be in the form: ( ) 1, when 1; 0 0, when 1, where: Weibull distribution therefore, initially the process is in the state S1. Using the determined, acceptable transitions between states ( Figure 3) the kernel of the semi-Markov process was defined. The has the following form: This matrix is an important characteristic of the semi-Markov process. Its nonzero elements ( ) ij Q t denote the probability of the process transitioning from state S i to state S j in a time not exceeding t. Since the distributions of conditional durations of individual operating states do not belong to any of the parametric distributions in this study, the distribution function could not be written using a formula. This made it impossible to determine the kernel of the process in a uniform and transparent manner. In such a situation, it is possible to use the theory of perturbed semi-Markov processes, allowing for the approximation of the distribution function. For this purpose, the values of individual elements of the probability transition matrix were first determined as the frequency of occurrence of individual states in the whole sample. Estimated elements p ij of the matrix P are shown in Table 4. Conditional probabilities p ij constitute the stationary distribution of the Markov chain embedded in the analysed process. The highest values refer to entering into state S1, which means the physical implementation of a transportation task, which is a natural consequence of its highest prevalence in the transport system. However, there is a more important element of the diagnostics, namely limit values of this chain. They are calculated through the equation: where: P -probability transition matrix p ij from state S i to state S j ; P -stationary probability vector p j of the Markov chain.
In the case of the examined process, for the 7-state model, stationary probabilities p j are determined through the following matrix equation: with the normalization condition: 1 2 3 4 5 6 7 1 p + p + p + p + p + p + p = , which is equivalent to the following system of equations: The calculations were made using the Statistica software (https://www.tibco.com/search#q=statistica). The results are presented in Table 5.
The next step was to calculate the expected conditional duration of the distinguished operating states of the semi-Markov process. The results are presented in Table 6.   Average conditional duration of individual states and values of probability transitions constituted the basis for calculation of average unconditional process states duration j T according to the equation: For this purpose, the following equations were solved: The obtained expected values of unconditional duration i T of individual operating states of the process are presented in Table 7.
The random variables T i have finite positive expected values. This makes it possible to calculate the probability limits P j for the semi-Markov process, based on the assumption: which comes down to solving the system of equations: , where: T T T ⋅ p + ⋅ p + ⋅ p .
The results are the limit probabilities of the semi-Markov process, the values of which are presented in Table 8.

Results and discussion
The calculated limit probabilities P j constitute an important operational characteristic. They represent the behaviour of the system over a longer period of time t → ∞. For the examined enterprise, the greatest value was found for the state indicating the performance of a transportation task, i.e., S1. Such a result indicates a high (over 54%) usage of means of transport in accordance with their intended use. This result can be considered satisfactory.
The remaining values are definitely lower and result from the implementation of processes accompanying physical distribution, such as handling works (state S4 -10%) or daily current maintenance (S4 -10%). Quite a large result (more than 10%) was found for the state S2 -readiness of vehicle with driver, which may mean better organization of the process and reduced waiting time for documents, persons, service, etc. States S6 and S7 related to repair and overhaul activities may also need to be investigated. They account for more than 20% in total and, as unfit states, have a strong influence on the technical readiness factor, which is determined as the sum of the respective probabilities of the reliability states (Rymarz et al. 2016). For the system under analysis, exploitability states range from S1 to S5. This makes it possible to determine the readiness of the system on the basis of the semi-Markov model according to the equation: 1 2 3 4 5 K P P P P P = + + + + .
The determined coefficient of technical readiness is K = 0.7957 and means that 80% of the time the tested system is ready to perform the task. This is a satisfactory result, but inspection of the causes of the high probability values for unfitness states (S6 and S7) and their correction may result in an increase in this coefficient.

Reliability function
In order to find the reliability function of the studied process, the theory of perturbed semi-Markov processes was used. It required the determination of set A of exploitability states of the studied process, which included states S1 to S5 and set A′ of unfitness states, which included states S6 and S7. Then, in accordance with (Grabski 2015;Grabski, Jaźwiński 2009), coefficient e i was calculated (Table 9) on the basis of the equation: Using the expected values 0 i m (Table 10) calculated from the sample using the Equation (24), coefficient m 0 was calculated: where: 0 i p means the stationary distribution of the Markov chain in the set A of states.
Then, using the equation: value e was calculated. Results obtained: e = 0.0819 and m 0 = 181.13 allowed for determination of the reliability function R(t) according to an approximate equation: which for the analysed process takes the form: A graph of reliability functions for the 7-state model is presented in Figure 4.
Based on the reliability function it is also possible to calculate the mean time to failure of the object using equation: Placing the approximate reliability function: This means that on average every 36 hours there is a failure that requires the intervention of a mechanic. Such a result confirms earlier concern related to high asymptotic indications for the states S6 and S7. Therefore, it is necessary to diagnose in detail the process of repairs carried out in the examined enterprise.

Conclusions
The paper presents an analysis of a real system of operations of means of transport together with a mathematical model allows us to study their level of readiness to carry out transportation tasks using the semi-Markov model. The calculated characteristics made it possible to diagnose the analysed system and assess the level of technical readiness. Areas, which are a source of increasing potential within the organization also need to be inspected in detail.
Moreover, the author's intention was to show that although the form of real observations, especially for such complex technical systems, often differs from parametric distributions, there are methods of examination allowing for the determination of technical readiness and reliability of such systems.
Determination of the reliability of transport systems is an important management component, allowing to asses and, if necessary, modify the adopted strategies. It enables fleet reediness surveying and adjusting maintenance and repair processes to the tasks performed. It must therefore be carried out correctly. The mathematical modelling of transport processes should be preceded by a thorough preliminary analysis covering both their qualitative characteristics and statistical research. In addition, the selected mathematical methods should be used only if the assumptions for their use are confirmed to be met.
The semi-Markov processes presented in the paper are not a popular tool for the evaluation of complex transport systems, therefore such examples are scarcely found in the  literature. The reason for this is that the form of real observations, especially for such multidimensional technical systems, often differs from parametric distributions, which makes the analysis much more difficult. As such studies are isolated instances and there is a small number of real examples, the method presented in this paper fills this gap to some extent, which was one of the main assumptions of the author. Moreover, the intention was to emphasize the necessity to carry out qualitative and quantitative evaluation of the collected empirical data before starting to estimate the model parameters.
As a result of the transport system analysis using the semi-Markov model, values of selected indicators were obtained, which were then used to determine the probability of vehicles staying in the distinguished operating states. Such a model allows for a qualitative and quantitative assessment of reliability, identification of weak links and areas requiring detailed control, which can result in an increase of the organization's potential.
If a higher level of detail needs to be achieved, the model may be further developed by distinguishing additional activities in each of the operating states. Such a solution will make it possible, for example, to identify the cause of the increased limit values for the vehicles being in service and repair states. Diagnostics of the transport system with the use of semi-Markov processes based on the forecasting of selected operating indicators may therefore constitute good support for evaluation and control processes in such enterprises.

Disclosure statement
The author declares that she has no relevant or material financial interests that relate to the research described in this paper.