ASSIGNMENT OF STOCHASTIC MODELS FOR THE DOMAIN OF PORT TERMINAL OPERATIONS

e paper presents the optimization possibilities of port terminal operations in order to generate maximum gain. Planning these processes is a demanding daily task taken on before dealing with port management, since transhipment operations, cargo loading and dispatching, maintenance and control of transhipment and transportation means are subject to hardly predictable and unexpectable stochastic conditions under which planning optimal terminal operations will include the examination of stochastic processes on the terminal. A model of states and transitions with gain and a model of optimal strategies in terminal management are set up. Furthermore, a model for determining the structure of transhipment equipment is developed. e devised models were adapted to the speci#cities of a port terminal and tested using the sample of a general cargo terminal.


Introduction
Sea ports are links in the chain of transporting goods from manufacturers to consumers. Paulauskas and Bentzen (2008) emphasize sea motorways as a part of the logistics chain. Considering the constant growth of transporting goods worldwide as well as ever more distinct competition between ports in a speci c region, it is exceptionally important that port management develops an optimal business strategy and congruent operating plans to draw cargo and achieve maximum e ciency in the existing facilities. Furthermore, this strategy should also involve development guidelines on port facilities for achieving maximum gain.
Considering short-term, seasonal and long-term oscillations of the quantity and type of goods, transport is a characteristically stochastic, i.e. random process, which complicates the engagement of port facilities considerably. Also, uncertain factors, such as vessel delay, weather conditions and mechanical equipment faults additionally complicate balancing operational processes and de ned plans. For example, Baublys (2007) considers problems such as the development of a probability model to determine the malfunction of the terminal, the determination of emergency situations in the terminal based on statistical data, the optimization of the e ect of failures on the operation of the terminal, the identi cation of con icting situations when making managerial decisions in the terminal. e aim of this paper is to examine the possibilities of the successful management of a port as a transport system through the application of methods and procedures for the stochastic process theory. e key feature of stochastic models adapted to port operations is to contribute to planning, organization, management (Liu et al. 2009) and control of processes in ports (Jaržemskis, Vasilis Vasiliauskas 2007). Although various stochastic models have been taught, developed and successfully applied in the production and service provision processes over the last decades (Česnauskis 2007;Chen, Zeng 2010;Hess, Hess 2010), the possibilities of sea port management using stochastic models are still insu ciently researched in scienti c literature. Baublys (2009) evaluates a technological process as a random process and assesses respective models. e author suggests a methodology for formalizing technological processes in the terminal and criteria for the optimal control and quality of the technological process. Machuca et al. (2007) explore the management of service operations. Kia et al. (2002) study port capacity by computer simulation. Cullinane et al. (2005) apply a mathematical programming approach to estimate the efciency of container port production. Cullinane (2002) investigates possible methods and their applications for productivity and e ciency modelling of ports and terminals. Even though the wide range of planning problems within shipping industry received signi cant atten-tion from researchers so far, there are still problems that have to be addressed, i.e. planning port operations under uncertainty. Port and ship operations contain considerable uncertainty due to weather conditions, mechanical problems and strikes, and thus optimization under uncertainty is an important eld within operation research, see the survey by Gendreau et al. (1996). Wentzel and Ovcharov (1986) elaborate Markov stochastic processes and the theory of queues providing numerous examples of solutions that incorporate Markov stochastic processes. Radmilović (1989) describes the operation of port transhipment and transportation means involving Markov discrete processes at the constant time and suggests the application of models that describe a technological process of direct and indirect cargo transhipment using a di erential equation system. e basic aim of the paper is the examination of stochastic processes in the port terminal de ned by Markov chains for the purposes of modelling. ree basic terminal states and a matrix of transition probabilities between states have been determined, on the basis of which the probabilities of certain states a er n transitions are obtained. Each transitional state causes a certain gain/loss and the aim is to determine the overall gain/loss a er n transitions. Furthermore, this work suggests and settles possible strategies for terminal management and comes up with a model that brings the optimal strategy that can serve as a good basis for port management while bringing tactical decisions. e article also helps with determining the best structure of transhipment means that assists in achieving maximum gain. For the purpose of testing the suggested models, daily work charts of the general cargo terminal were analyzed. e obtained results provided initial data on forming a matrix of transition probabilities, state vectors and a gain matrix. On the following pages, there is a short description of the problem followed by the models for nding a solution to the problem and results of the test made for a general cargo terminal case in the port of Rijeka. Finally, the bene ts and shortfalls of the given models along with practical applications are highlighted as well as the possibilities of the further development of the suggested models are displayed.

e Problem
A port terminal is one of port subsystems where the operations of vessel transhipment and unloading, cargo loading and dispatching, transhipment and transportation equipment (TTE) control and maintenance take place (Hess et al. 2008). Daily running of the above mentioned operations are a ected by stochastic factors that are hard or impossible to predict and which disturb normal running of operations according to the operating business plan (Hess, Hess 2010;Vukadinović, Popović 1989). ese factors are caused by weather conditions, vessel delays, cargo delays in land, market uctuations, worker strikes, breakdowns due to port equipment faults, etc.
In case the terminal is equipped with new and reliable equipment that is rarely damaged, the probability of the intermission state due to maintenance would be relatively low. Likewise, if a terminal is extremely busy with respect to cargo unloading and loading, then the probability that this terminal would be in the standby state will be insigni cantly low in relation to the operating state (Hess et al. 2007).
When beginning with the initial state, the main problem is to determine the probabilities of certain states a er a certain number of phases in order to undertake necessary actions that would influence the change of states at a certain moment in the future or actions that in uence the adjustment in the state to come in advance. Along with the probabilities of certain states, it is extremely important to determine the overall gain of terminal operation. For that purpose, a model of gain was developed in this work. Furthermore, by setting possible business strategies, the goal is to determine an optimal strategy that yields maximum gain (Vasilis Vasiliauskas, Barysienė 2008). erefore, a model for optimal strategies was developed. Another problem was how to determine optimal structures of TTE on a terminal accomplishing maximum operating and economic results. e problem was solved around the existing terminal resource structure and a suggestion was given for model modi cation to encompass changing conditions in the market of TTE.

Model Setup
Considering operating processes in the port terminal (Hess, Hess 2009), three basic states when a terminal can be found at a given moment can be distinguished: S 1 -standby state (no working operation on the wharf, but the collection and analysis of data regarding cargo, vessel and weather); S 2 -operating state (cargo loading and unloading, cargo dispatching from the operating wharf to warehouses or inland means of transport; this is the most favourable state economically); S 3 -intermission state (regular maintenance of TTE, repairs in case of a sudden fault, break in case of bad weather which prevents from safe transshipment actions, break due to possible workers' strikes, postponement of work due to vessel or cargo delay from the inland). e following possible transitions between states were determined from the above mentioned ones: I 12 -from the standby state to the operating state a er vessel arrival, cessation; I 13 -from the standby state to the intermission state; I 21 -from the operating state to the standby state; I 23 -from the operating state to the intermission state; I 31 -from the intermission state to the standby state; I 32 -from the intermission state to the operating state a er eliminating causes for breakdown. Port terminals act stochastically if states and transitions between individual states do not follow the course of operations due to various internal and external unpredictable e ects on regular operations. erefore, a terminal will be represented as homogenous Markov chain with the following matrix of transitional probabilities: (1) and initial state vector P 0 : ese formulae will be implemented in the experiment presented in Part 4.

States and Transitions with Gain
Terminal processes, de ned by states and transitions, have economic e ects since all transitions between states cause certain gain or loss. Let us assume that r ij is the gain caused by system transition from state x i to state x j and interpreted as: gain for direct transition; gain for being in state x i (or x j ) throughout a single time unit. e interpretation of gain for direct transition can be applied to shipping where individual ports make for system states, transition between states makes for goods or passenger transportation between ports and r ij is the ship owner's gain for the carried transport. e second interpretation of gain discussed in this paper contributes to practical application; thus, terminal states are de ned by terminal standby, operating and intermission states (including transshipping and transportation means and workers), so that r ij is relevant pro t gained while the terminal is in state x i before transition to state x j . If r ij > 0, the terminal is exploited and gains pro t; if r ij < 0, the terminal is in the standby or intermission state and negative gain which is loss is achieved. e task given in this paper is to come up with a model to nd overall gain a er n terminal transitions between states.
Let us assume that a system, in which a discrete Markov process is taking place within discrete time (Markov chain), has k states, and the matrix of transitional probabilities P and the corresponding matrix of gain R are (Vukadinović, Popović 1989): Let us assume that matrix R is symmetrical, i.e. r ij = r ji . e following assumptions are given: the system begins to function from state x i ; overall gain a er n phases (transitions) is equal to v i (n); gain for one phase from state x i to state x j equals r ij ; gain for n phases can be represented as the sum of r ij +v j (n-1), where v j (n-1) is gain for n-1 transitions starting with state x j . System transition from state x i to state x j is accomplished by probability p ij , so the expected overall gain for n phases of the system starting with functioning from state x i , equals: If we introduce symbol q i for the expected gain for one phase from state x i to x j , state (4) can be written in the following form: Gain measures v 1 (n), v 2 (n),..., v k (n) are the components of the gain vector for n phases, v (n) = (v 1 (n), v 2 (n), …, v k (n)), whereas gain measures q 1 , q 2 ,..., q k form the gain vector for one phase: q = (q 1 , q 2 , ..., q k ). e components of vector q can be noted in matrix products P and R, i.e. in matrix G: Considering the condition that r ij = r ji , it is clear that the components of vector q form the main diagonal of matrix G. Equation (4) can be written in the following form of the vector: If there are stationary possibilities for Markov chain, i.e. if Markov chain is ergodic, than for n → ∞: e expected gain in the stationary regimes of system operation equals:

Optimal Strategies for Terminal Operations
A general approach to solving problems of operating process management that can be de ned by Markov chains is comprised of examining k of di erent rules (strategies) leading to appropriate solutions. Alternative solutions are obtained by changing the elements of matrixes P or R. For the h-th rule, let us mark matrices P, R and G with h and their elements as follows:`^`^` Maximum expected gain for n phases, if the system is initially in state x i and takes on optimal value in each of the following transitions, equals: where: v j (n-1) is the maximum expected gain for n-1 phases if the system started functioning from state x j . In the matrix form, (11) is: For a su ciently high n number of transitions between states, it is suitable to introduce multiplier E (0 d E d 1) that multiplies expression P (h) • v(n -1) in Formula (12). Since this proves to be suitable practically, this multiplier allows maximum gain to always be nal. e selection of an optimal solution is done as follows. In order to determine the optimal solution at the rst stage, i.e. for n = 1, the initial state of gain is de ned: v(0) = 0 and matrices By selecting elements on the main diagonal from each G (h) matrix, the result of h vector-columns is obtained: (1) All these vectors can be collected in one rectangular matrix (generally h z k): § ·¸ Product P (h) • v(n -1) is calculated and, since the expected gain for the initial state is v(0) = 0 and represents the null vector, P (h) • v(0) = 0. In order to determine maximum gain at the rst stage, it is necessary to select a maximum element of vector-row from matrix (15) and to determine it:^`^` is way, a procedure for determining an optimal variant at the rst stage is reduced to an overview of the vector-row element values of matrix ( ) h k q and maximum selection. en, a vector is formed from the symbols of maximum element points: where: the i-th element d i (n) is the whole number between 1 and h, which shows the ordinal number of a rule and maximizes the expected gain for one transition if the system starts functioning from state x i . Determining optimal solutions at the second stage (n=2) is as follows. e calculation procedure is analogue to that at the rst stage. e initial data is comprised of matrix ( ) h k q and the vector-column of calculated gains v(1). e procedure is similar to that at stages n = 3, 4, ... At the same time, a set of vectors of optimal rules for operational decisions is obtained.

Experiment and Results of Analysis
e elements of the transition probability matrix are obtained by a statistical analysis of processes from daily work charts on the examined terminal in the port of Rijeka within one year (2009) and are interpreted through running daily operations. In the observed period, the terminal is found in the standby state with the probability of 0.55 due to considerable capacity unemployment. Transitions from the standby to intermission state occur with the probability of 0.05 because of regular equipment maintenance. Since the existent operating means on the operating wharf of the general cargo terminal are outdated, the probability of transition from the intermission state to the operating state is 0.2. From the intermission mode, the terminal returns to the operating state with the probability of 0.75, since the intermission mode most o en occurs in the event of fault for mechanical equipment during transhipment/transportation. Due to the speci c organization of work and semimechanized work, inadequate equipment for machine maintenance and probability that the terminal will again return to the intermission state in the next phase is 0.2.
From nancial reports based on daily work charts on the examined terminal, gain matrix R was extracted and expressed as a coe cient: § · ¸ ¨ © ¹ If the terminal remains in the standby mode, it does not gain but loses due to the ine ciency of facilities with a coe cient of -1. By transition from the standby state to the operating state, the terminal gains the pro t of +1. By transition from the standby state to intermission state, the terminal will lose more than in the standby state because of maintenance expenses (-1.5). Gain is the highest in the operating state and totals 2. In case it is necessary to switch from the operating to intermission state, the terminal will neither lose nor gain. e terminal returns from the intermission to operating state and then neither gains nor loses. Remaining in the intermission state is obviously the most unfavourable (loss -2.5) because the terminal does not earn any money but spends on recovering from intermission.
For each individual case in practice, values from gain matrix R can be turned into certain money units. It is important to understand relations between states that are the nature of transition between states (whether transition is positive, negative, higher or lower in relation to another regarding pro t gain). en, these values can easily be turned into money units for the speci c case (for example, USD 1 000 or HRK 1 000). From (6) From (7), we then obtain gain vector for one stage starting from the rst, second and third state: e obtained results have the following meaning: if the terminal was initially in the standby state, following one transition, there would be a loss of 0.225 money units (m.u.). e reason for this is the fact that the next transition will have a relatively high probability of 55% to return to the standby state where the terminal gener-ates pro t. If it was in the operating state, the generated pro t would make 1.12 m.u., which is even higher if the probability of transition back to the operating state is higher than 40%. Starting with the intermission state which per se generates loss for the terminal, any further transition will generate yet another loss; only this loss will be lower if transition is to the operating state. Considering the probability of transition to the operating state of 75% and the probability of remaining in the same intermission state of 20%, the generated loss a er one transition is 0.575 m.u. e expected gain following two and three stages is: § ·¸ If at the beginning of examination the terminal was in the standby state, a gain of 0.43 m.u. was generated following three stages, which can be explained by the fact that transition to the operating state, in which the terminal generates pro t, was achieved in three transitions with the probability of 40%. e pro t of 1.87 m.u. generated within three transitions in case the terminal was initially in the operating state, is not much higher than that generated following the rst stage (1.2 m.u.) precisely because of similar probabilities of transition (40%) to the unfavourable standby state and operating state. Furthermore, a gain in case of transition from the intermission state is 0.58 m.u. and is not that negative as it was following the rst stage. is is due to high probability that the terminal will move from the intermission state to the operating state at three stages.
In order to determine the ultimate expected gain, rst, it is necessary to nd the nal vector of state probability (vector or ergodic state probabilities) t = (t 1 , t 2 , t 3 ) from the condition of t u P = t : § ·  With (9), the expected pro t was generated in the stationary operation regime q = 0.3656, which means that the expected gain, when examining the terminal during a larger number of stages with the known current state, equals 0.3656 m.u. is is valid regardless of the current terminal state and the state at which the terminal will move to the next transition.

Optimal Strategy Model for General Cargo Terminal Management
e previous section shows it is necessary to improve the e ciency of business in the general cargo terminal in the port of Rijeka. e question raised is which measures should be taken and the optimal strategy selected out of the possible ones devised for that purpose: I. to draw cargo, i.e. increase throughput without any investments in new equipment supply, enhance or renovate the existing facilities (throughput n; investments o), II. to increase throughput and invest nancial means in order to provide faster and better service on the terminal (throughput n; investments n), III. throughput remains the same and means are invested in more e cient service providing (throughput o; investments n ). Starting with transition probabilities, matrix P and gain matrix R set up in the previous section of the experiment, each of the mentioned strategies, considering its content, draw associated probability and gain matrices.
Having the known matrices P (h) and R (h) , a er a short calculation of (13) and (14), q (h) is determined, and considering that v(0) =0, equation (12) where maximization for three elements of the rst, second and third line is done. Accordingly, the associated vector of the expected maximum gains: § ·¸ Finally, the vector of optimal rules is: e results show that in practice, if the terminal is in the standby state, rule h = 1 or h = 2 should be used (in all respects, to increase throughput by investing or not investing in modernization). In that case, the maximum expected gain is 0.475 money units. If the terminal is in the operating state, rule h = 2 should be used (to draw cargo, but through investments in port facilities) and then the expected maximum gain is 3 m.u. e second rule h = 2 should also be applied to the intermission state where the maximum pro t generated is 0.15 m.u. e following is obtained with (12)  e procedure is the same for stages n = 3, 4, ... Calculations are done until the desirable number of transition is achieved. At the same time, a vector set of optimal rules is noted until this vector is stabilized. is means that the calculation procedure should proceed until state vector stops changing so that at stages n it equals state vector following stages n -1, which in the case of the examined terminal was achieved for n = 8 (see 4.1). us, the obtained solution denotes that there is a single determined rule of making a decision on all states, and that this rule should be followed in the course of time.

Model for an Optimal Structure of TTE on the General Cargo Terminal
Following a decision to apply a strategy for increasing capacities regarding the modernization and supply of new TTE, a question is raised as to which type of resources to obtain should be the most cost-e ective.
is issue should be addressed with regard to the current state of equipment on the terminal the structure of which consists of diesel forkli s and makes 31%, trailer tugs -23% and mobile cranes -46%. e vector of the initial state probability is: 32) and the matrix of transition from state i to state j, that is transition from one type of resource to another obtained from the comparative analysis of operating e ects of the existing equipment and professional opinion, is: Let us assume that the supply cycle (phase duration), following which the changes of the state are recorded, is 5 years. State probabilities a er n phases are obtained by P (n) = P 0 • P n , which means that P is means that following 3 stages (15 years), the structure of TTE on the operating wharf of the general cargo terminal in the port of Rijeka would make 26.3% of diesel forkli s, 20.4% of trailer tugs and 53.3% of mobile cranes.
In order to apply to value indicators in the previous example that bring us to the optimal resource structure, the gain matrix is constructed, which is, considering the overall expenses of supply and maintenance, is obtained so that the overall expenses of supply and maintenance for three above mentioned types of means were put in the following relation: D:T:M = 4.99:4.85:2.44, where: D, T and M are symbols for the quantity of diesel forkli s, tugs and mobile cranes respectively. e values of relation are associated money units. If the initial assumption is that the e ect of remaining with mobile cranes makes 8.0 m.u., other e ects are computed referring to the obtained ratio and percentages of increasing/lowering expenses generated by transitions to other types. erefore, the gain matrix is: which means that the best e ect is achieved using mobile cranes, that is transition or remaining with that type of means in order to perform the greatest part of operations in the terminal. is was well expected since the overall supply and maintenance expenses, with regard to the operating e ects of mobile cranes, are the lowest ones. e presented model was also extended to the modi ed content of the TTE supply problem. For example, a new type of equipment along with the existing ones may appear on the market. It is assumed that this type would be given priority over the other ones. Complete production cancellation of some existing equipment becoming technologically outdated might also take place. erefore, the existing structure can change by introducing a new type of equipment and/or by eliminating the existing one.
In the case of introducing a new type of equipment, the vector of the initial state probability would be P 0 = [0.31 0.23 0.46 0.00], where element p 4 = 0, since at the beginning of testing the structure, there was no Type 4 of equipment, which appeared in the following 5 years. en, the probability matrix of transition with the introduced Type 4 or equipment is: e matrix shows that transitions from all types of equipment to the new ones are the most probable, along with the probabilities of remaining on the same type due to established practices and a lack of justi cation for transition to another type of equipment or obtaining a new type if the old one meets demands. Following stage 1 (5 years), vector P (1) (state probabilities) is P (1) = [0.185 0.054 0.407 0.354] and state probabilities following the second, that is stage 3, make P Even following stage 1, the introduced new type of equipment in the overall structure represents 35%, following 10 years -57%, and following -15 years, i.e. 3 steps, the ratio of a new type of equipment will make 71%, whereas the ratio of other types is insigni cant, with the exception of the mobile crane making approximately 21%. Such a high ration of the new type of equipment can be explained by constant advances in technology and new solutions to higher e ects and lower maintenance expenses.
In the case of eliminating the existing type of TTE, the vector of the initial state probability is the same as in the case of introducing a new type since this existing equipment constitute only 23% of the structure in the beginning. erefore, P 0 = [0.31 0.23 0.46 0.00]. However, the transition probability matrix, which is now: In transition probability matrix P, all probabilities in the second column equal zero, which means that transition to the second type of TTE is impossible. For this reason, this type of TTE is not present in the TTE structure following stages 1, 2, 3, etc. e structure of TTE following stage 2, i.e. 10 years, during which the elimination of the existing and introduction of a new type of equipment took place, makes 13% of the rst, 0% of the second, 29% of the third, and 58% of the new type, and following 3 stages (15 years), there is 8% of the rst, 20% of the third, and 72% of the new type of TTE.

Conclusions
1. e main aim of this paper was to examine the possibilities of successful management of a port as a transport system through applying the methods and procedures of the stochastic process theory. 2. e advantage of the model set with gain is in the quanti cation of transition probabilities which can be expressed in any measurable units. e coe cients from gain matrices calculated considering nancial reports have a direct in uence on the nal results; thus, the selection and quality of data obtained from work datasheets are of particular signi cance and may represent the object of another research. is paper examines transitions generating loss and pro t for the terminal. e practical use of the model for port management lies in the simplicity of procedure applications in order to determine and plan measures to increase the probabilities of the most favourable transitions with regard to generating the greatest pro t. 3. e set up model of Rijeka general cargo terminal provides an answer to the question regarding the most optimal business strategy under the given conditions. e results of the examined terminal showed that the rst and second strategies with an increase in throughput and investing, i.e. no investments in facilities, were equally good. If examination starts with the operating state, it is necessary to use the second strategy, i.e. drawing cargo and investing in facilities. e same strategy should be applied if the terminal was initially in the intermission state. 4. A er making a decision on how to apply the strategy for increasing capacities regarding the modernization and supply of new TTE, a question as to which type of equipment to obtain would be the most cost-e ective is raised. It was identi ed that when following each stage, the best e ect was achieved using mobile cranes, i.e. transition to or remaining with this type of equipment to perform the greatest part of operations on the terminal. is was well expected since the overall supply and maintenance expenses, with regard to the operating e ects of mobile cranes, are the lowest ones. Also, the possibilities of model expansion considering possible changes in the long term on the TTE market were examined. 5. Further research should be directed to the development of models within the meaning of educating an additional operating process in a wider port area. e expansion of gain models would be classifying gain with regard to various types of cargo in transhipment, or various types of vessels. When dealing with the model of optimal strategies in terminal business, it is possible to develop additional strategies with more detailed content.