PREDICTION OF FOREX TREND MOVEMENT USING LINEAR REGRESSION LINE , TWO-STAGE OF MULTI-LAYER PERCEPTRON AND DYNAMIC TIME WARPING ALGORITHMS

Foreign Exchange Currency prediction has become a challenging task since the late 1970s due to uncertainty movement of exchange rates. However, most researchers in this area were neglecting to analyse trend patterns from historical Forex data as input features. Thus, this motivates us to investigate possibility of repeated trend patterns from historical Forex data. This paper aims to investigate the repeated trend patterns as features from historical Forex data, which proposes new combination techniques Linear Regression Line, two-stage of Multi-Layer Perceptron and Dynamic Time Warping algorithms in order to improve the performance of prediction significantly, thus achieving greater accuracy.


INTRODUCTION
Currency investments are affected by factors such as erroneous prediction, information and estimation of time; hence many investors are faced with uncertainty in their investments in the past decade.Therefore, the request has been made for prediction techniques to get better odds of profit from investments.The price in any exchange never represents the fair value in financial market due to variations in supply and demand from investors.Thus, Foreign Exchange Currency (Forex) and stock prediction has become a challenging task to the Artificial Intelligence (AI) community.
Generally, the most common issue in Forex and stock prediction is that most investors are usually unaware of the behaviour and patterns of price within historical data.Investors only know that the progress of Forex and stock market depend on relevant news from newspaper, magazines and other resources (Falinouss, 2007).Researchers from the financial sector have only used technical analysis methods to analyse trends for the prediction of trend movement (Dzikevičius & Šaranda, 2010;Maknickienė & Maknickas, 2012;Tanaka-Yamawaki & Tokuoka, 2007).This is because the obtained time series data does not contain sufficient information to understand the future trends.Therefore, there is high level of difficulty in predicting the financial time series data.The implementation of trend patterns as a prediction factor is suggested for improving the existing prediction result.A unique cluster of trend patterns can be discovered by observing the historical data.
Currently, the reliability of financial time series prediction model is not satisfactory, due to the lack of effective data analysis of trends over a period of time (Falinouss, 2007).Researchers may not fully utilise the available information and reduce the noise present for predicting the trend movement.Therefore, the accuracy of the prediction model is unsatisfactory and unreliable.It is difficult to determine the usefulness of information from the financial time series data.
Researchers from computer science sector have applied AI techniques such as Artificial Neural Network (ANN) algorithm (Desai, Trivedi, & Joshi, 2013), Expert Systems (Lee & Jo, 1999), Hidden Markov Model (HMM) (Gupta & Dhingra, 2012) and Genetic Algorithm (Hassan, Nath, & Kirley, 2007) as classifiers in financial market to learn and predict the prices for improving the prediction accuracy.However, these methods still inevitably have their own limitations.
Thus, this paper aims to investigate the repeat behaviour and pattern of trends from the historical financial time series data, and utilise the strength of machine learning techniques to develop a promising financial time series predictor engine.In our proposed approach, an identifier Linear Regression Line (LRL) that has been utilised in order to analyse the trend patterns and formed general patterns of Forex trends.LRL is a statistical tool that uses the slopes' value of regression to identify the distance between the prices of timeline and the trendline.Then, those trend patterns were collected and used as inputs to train Multi-Layer Perceptron (MLP) for classification.Hence, MLP is a supervised learning technique which has the ability to learn the patterns stochastically and generate its own knowledge from the surroundings.For predicting the future trend movement, Dynamic Time Warping (DTW) is used to measure the similarity based on the trained models.DTW is a time-series alignment algorithm for recognising and comparing different patterns by warping the distance.
The paper is organised as follows.Next section describes the Background and Related Work for financial time series prediction and machine learning techniques.Proposed Prediction Model section describes the proposed model and each process in detail, including the segmentation algorithm.Segmentation, clustering and prediction results are discussed in Experimental Results section.Finally, Conclusion and Future Work section presents our conclusions and future work of this proposed prediction model.

LITERATURE REVIEW
Since the past forty years, there are a large amount of intensive researches have been discovered for financial time series prediction.Lee and Jo (1999) applied candlestick pattern to develop an Expert System to predict the timing to buy or sell stock.They proved that the system able to provide the decision of stock price to investors, and also concluded that the limitation of their system was lack of automated learning.Person (2002) conducted an experiment by using candlestick pattern with two technical analysis methods known as Exponential Moving Average (EMA) and Moving Average Convergence-Divergence (MACD) for predicting the stock trend.Also, Lee and Liu (2006) proposed a financial decision model that combined candlestick pattern with fuzzy linguistic variables, and designed an information agent to collect the financial time series data.Both authors concluded that candlestick pattern helps traders to interpret the changing direction of financial trends.
Lawerence (1997) utilised the technical analysis method -EMH (Efficient Market Hypothesis) as a source feature for ANN to test for stock price, and found that the features of stock trend must be fully understood so that ANN will learn the correct pattern for prediction.Also, Zhai, Hsu and Halgamuge (2007) proposed a prediction model that combined the related news with few technical indicators to analyse the stock trends for Support Vector Machine (SVM) to train and predict the direction of stock price.The authors found that the prediction model only performed better when certain variable values is static.However, Shmatov (2012) found that technical analysis methods are able to provide the trend patterns for machine learning algorithms to recognise the patterns for predicting the trend movement.Desai, Trivedi & Joshi (2013) implemented data pre-processing technique and ANN in order to analyse the trend patterns for prediction.Their experiments proved that ANN has the ability to learn hidden trend patterns and predict the direction of the stock trends.Other experiments were done by Charkha (2008), Tiong, Ngo and Lee (2013), Maknickienė and Maknickas (2012), which have shown significant results through implementing ANN, and proved that, ANN is more consistent in recognising the patterns of stock price and Forex exchange rates.Another experiment conducted by Gupta and Dhingra (2012) that used HMM with maximum posteriori estimator.The performance of their approach was only well-performed via certain selected variables.
ANN is a mathematical model that inspired by the neural network that forms the human brain.It consists of an input layer, hidden layer and output layer (Alvarez, 2006).During the learning and training process, the training inputs are presented at the input layer, with connections within the hidden layer used to acquire classification, while the desired output is presented at output layer (Kamruzzaman & Sarker, 2003).In other words, ANN acts as a black box to classify an output based on patterns recognised from a given input.Thus, the ANN with Multi-Layer Perceptron (MLP) algorithm is widely used as classifier, which were carried out from WEKA open source library.Many studies have examined that the learning performance of ANN is flexible compared to other methods (Kamijo & Tanigawa, 1990;Charkha, 2008;Kamruzzaman & Sarker, 2004;Enke, Grauer, & Mehdiyev, 2011;Desai et al., 2013).Also, the features representation is a critical factor in the performance of ANN because it contains important hidden information in order to train the ANN consistently for predicting the future trends.
Hypothetically, the trend pattern of a financial time series data repeats itself at different time stamp.Linear Regression Line (LRL) as statistical tool which is widely been used as features to represent the trend patterns (Rinehart, 2003).Also, it is used to calculate the slope value of regression line: a straight line is drawn through the time-series data to identify the distances between the prices of different time frame and trendlines (Ahangar, Yahyazadehfar, & Pournaghsband, 2010).As well, LRL calculates a "best-fit" line through the financial time series and provides a slope to identify the trends.It also has the ability to avoid the unbiased fit in the data.
DTW is an algorithm that measures the optimal alignment between two time series data by warping the distance.The warping between two time series data can be used to determine the similarity between themselves (Albrecht, 2009).Euclidean distance is an efficient distance measurement which calculates the sum of squared distance from each point of one time series to other point of time series data (Niels, 2004).Therefore, DTW has the ability to find the best match between two sequences of time series data.
In conclusion, the reliability of prediction model is not satisfactory, due to the lack of effective data analysis of trends over a period of time.As such, researchers may not fully utilise the available information for prediction.Therefore, the accuracy of the prediction model is unsatisfactory and unreliable.Thus, this research applied the proposed features by using LRL and proposed segmentation algorithm which would be explained in details as follow section, then supported by ANN-MLP algorithm as classifier and DTW algorithm for prediction in order to improve the accuracy of prediction model.

PROPOSED PREDICTION MODEL
This section proposes a prediction model using MLP, supported by LRL and DTW.Fig. 1 illustrates the process flow of our proposed model.Initially, LRL is applied to analyse and form the general trend patterns.Followed by, our proposed Segmentation Rules is implemented in order to extract the different types of trend patterns.However, the implementation of two-stage of MLP is used to learn the trend patterns for trend classification.In the first stage of MLP, it was used to learn the two archetypes of trend patterns based on the proposed features, which will be briefed in details.Then, second-stage of MLP is applied in order to recognise the specific types of trend patterns.

Forex Data
Analysis Feature Extraction

2-Stage of Learning Process Prediction
Lastly, DTW is used to determine the degree of similarity based on the trained model.This technique is used to compare the unknown partial patterns against the reference patterns from database, in order to measure the similarity between the patterns for predicting the trend patterns.The framework of this approach will be discussed in the following sections.The model is divided into three sections: Pattern Analysis, Learning Process and Prediction.

Pattern Analysis
In this section, we selected the historical Forex data AUD -USD and EUR -USD from the beginning of 2012 until the end of 2012 in Free Forex Historical Data website (Free Forex Historical Data, 2011).Australian Dollar (AUD), Euro (EUR) and U.S. Dollar (USD) are the world's largest currencies, and many multinational corporations conduct business in the Australia, Europe and United States.Many investors are often attracted by the exchange rate that offers very low bid and constant liquidity.
LRL is implemented to analyse the dataset, representing the trend in the form of a pattern, by mean of using the slopes' value of regression to identify the distance between the prices of timeline and the trendline.The slope is used to identify trends; a positive slope is defined as an uptrend whilst a negative slope is defined as a downtrend (Rockefeller, 2011).The following equation defines a straight line to describe the trend: where y is defined as closing price, m is defined as slope, x is defined as the number of time stamp and b is defined as y intercept.
Eq. ( 1) had been applied to derive a straight line through the exchange rates from different time stamps for identifying the overall trend.Furthermore, we implemented Segmentation Rules from Fig. 2 to extract the trend patterns.
The Segmentation Rules start from the beginning of the data to discover the exchange point from the different types of trend patterns.The size of Forex data is grouped in weekly format.Once the patterns are fitting the either one of archetypes shown in Fig. 3, the values of the resultant trend vector will be stored in the database.
After the segmentation, k-mean clustering algorithm (Jain, Murty, & Flynn, 2000) had been applied to cluster the trend patterns into two main clusters -"Uptrend" and "Downtrend".In each cluster, we implemented k-mean clustering algorithm to identify the patterns as "Normal", "Skew Left" and "Skew Right".We utilised the selected features which were shown in Fig. 4 to represent the trend patterns for clustering.
Input: Historical Financial Time Series Data T Output: Trend patterns

Figure 2. Segmentation Rules
According to the features that are listed in Fig. 4, "Feature 1", "Feature 2", "Feature 4" and "Feature 5" are utilised to represent the trend patterns.To assign the values for these 4 features, value 1 represented the slope as up direction, 2 defined as unchanged direction, and value 3 defined as down direction."Feature 3" and "Feature 6" were used to identify the areas of trend patterns."Feature 7" calculated the length of trend patterns.

Learning Process
Once the previous section had successfully segmented the trend patterns, we utilised MLP algorithm to learn the trend patterns based on the feature representation in Fig. 3 as input parameters.MLP is feedforward ANN, which consist of multiple layers of nodes in a directed graph (Principe, Euliano, & Lefebvre, 2000).All the nodes of each layer are connected to each node in the next layer.The inputs are presented at the input layer, with connections within the hidden layer used to acquire classification, while the desired output is presented at output layer.The following equation defines input parameters to describe the trend: represented the slope as up direction, 2 defined as unchanged direction, and value 3 defined as dow direction."Feature 3" and "Feature 6" were used to identify the areas of trend patterns."Feature 7 calculated the length of trend patterns.9 presented the slope as up direction, 2 defined as unchanged direction, and value 3 defined as down irection."Feature 3" and "Feature 6" were used to identify the areas of trend patterns."Feature 7" alculated the length of trend patterns.(2) where denoted as trend patterns, x 1 defined as "Feature 1" and x n defined as "Feature 7".
Furthermore, this equation defines as MLP processing to classify the patterns: (3) where y defined as output, nonlin is an activation or nonlinear mapping function, w denoted as the weights, x is the pattern features and b is a bias.
In this learning process, we applied two-stage of MLP in order to enhance MLP algorithm, which able to classify the proposed trend patterns more specific.Fig. 5 illustrated the first stage of MLP learning process.We implemented Eq. ( 3) from WEKA open source library to determine two main clusters -"Uptrend" and "Downtrend" by giving input parameters based on Eq. ( 2), in order to learn the trend patterns.Upon the first-stage of MLP processing was done, we applied second-stage of MLP to learn different types of "Normal", "Skew Left" and "Skew Right" patterns in each of these clusters.In order to calculate the minimum error of training process, Mean Absolute Error (MAE) and Root Mean Square Error (RMSE) were used in this model.
Furthermore, this equation defines as MLP processing to classify the patterns: where y defined as output, nonlin is an activation or nonlinear mapping function, w denoted as the weights, x is the pattern features and b is a bias.
In this learning process, we applied two-stage of MLP in order to enhance MLP algorithm, which able to classify the proposed trend patterns more specific.Fig. 5 illustrated the first stage of MLP learning process.We implemented Eq. ( 3) from WEKA open source library to determine two main clusters -"Uptrend" and "Downtrend" by giving input parameters based on Eq. ( 2), in order to learn the trend patterns.

Prediction
To predict the future exchange rate trends, we had utilised the two-stage of MLP model to classify an unknown partial patterns belong to the specific cluster of trend patterns, and applied DTW to find the warping path between the patterns.DTW is a time-series alignment algorithm for measuring two sequences of vector values by warping the distance until an optimal match between the sequences (Albrecht, 2009).Euclidean measurement is an efficient measurement which calculates the sum of squared distance from points to points in finding the distance path.
DTW firstly initialises a matrix table and calculates the accumulated cost matrix by using a distance measurement function.The following equation shows the idea: where W is defined as a pattern vector values, Dist(W) is the distance of warp path for W, Dist(w ki , w kj ) is the distance between two set of patterns values, k is defined as number of warp path between two patterns, w is defined as time series data, i and j are defined as an index from different set of time series data.
Based on the output from cost matrix table, DTW identified the warping path between unknown and reference patterns by using Eq. ( 4).Then, the remaining sequence elements between the two trend patterns are used for predicting the future of trend movement.The result obtained in the matrix table from Fig. 6 showed a warping path between the patterns.However, the result does not provides the unmatched information between the patterns.To identify the unmatched vector from the table, we introduced Kernal-based Tracing algorithm.

Fig. 7 describes the Kernel-based Tracing algorithm to identify the diagonal in the cost matrix representation table. The cost matrix representation table
shows the final result of warping path between two trend patterns.Kernelbased algorithm is designed by 2 × 2 dimention array to identify the similiarity of two patterns.This algorithm assigns the vector values of warping path as 1 -starting point, 2, 3 -unmatched, and 4 -matched.Once the matched part is found in diagonal, it could identify the last vector point of partial pattern is matched at any vector point of full pattern.The remaining sequence elements from the cost matrix table are utilised to predict the future of financial trends.

RESULTS
In our experiment, we have selected the historical data of Currency Exchange AUD -USD and EUR -USD from the beginning of 2012 until the end of 2012 in Free Forex Historical Data website (Free Forex Historical Data, 2011).The Forex data are presented in weekly format with half-hourly intervals.
Based on the historical financial time series data, Eq. ( 1) has been utilised to analyse the overall trend.This equation only concentrates on the close price from the data to analyse the financial trends.Fig(s).8 to 11 show certain experiment results of overall trend patterns using Eq. ( 1).According to these results, there are certain trend patterns as mentioned are repeated itself in different time stamps.The trend patterns could provide an idea of the direction in which an investment's value might move.The results had discovered that LRL could analyse the overall trend patterns from the historical Forex data with a clear and smoothness results.
In the following steps, segmentation rules (see Fig. 2) are implemented to identify as two general archetypes of trend patterns shown in Fig. 3.Moreover, k-mean algorithm is utilised to group the trend patterns in the database, so as to increase the performance of machine learning algorithms during the learning process.Table 1 presented the clustering results.In the following steps, segmentation rules (see Fig. 2) are implemented to identify as two general archetypes of trend patterns shown in Fig. 3.Moreover, k-mean algorithm is utilised to group the trend patterns in the database, so as to increase the performance of machine learning algorithms during the learning process.Table 1 presented the clustering results.
(continued) Based on the results, the selected features have successfully represented the trend patterns which allowed k-mean algorithm to group the patterns into two general archetypes that were shown in Fig. 3.The different types of "Uptrend" and "Downtrend" patterns are clustered into "Normal", "Skew Left" and "Skew Right" that are shown in Fig(s).12 and 13.Then, the second-stage of MLP classifier could learn the trend patterns more efficiency in classification.Since the pattern analysis stage had successfully analysed the exchange rates trend patterns, we utilised two-stage of MLP as classifier to learn the trend patterns.From the results of two-stage of MLP classification which shown in Tables 2 and 3, we used MAE and RMSE to prove that our two-stage of MLP model can classify the patterns with the selected features.According to the results, the two-stage of MLP model is well-trained in identifying the trend patterns.To classify that the performance of two-stage of MLP model, we selected the new historical partial data of AUD -USD and EUR -USD from the beginning of January 2013 until the end of March 2013, which defined as unknown patterns.In the experiments, our model had randomly selected 80 set of unknown patterns for classification.The rates of classification shown in Table 4 provide 100% accuracy.This study proves that the proposed features (see Fig. 9) could be a better preference in representing the trend patterns for MLP to learn.The experiment results shown in Table 4 also proved that MLP model was well-performed in classifying the unknown patterns that were belong to its specific clusters of trend patterns based on the 7 features.Note.* The dataset contained 3 types of trend patterns -"Normal", "Skew Left" and "Skew Right" from "Uptrend" and "Downtrend" groups.In Experiment 1, the model has predicted 28 successes out of 40 attempts of unknown patterns.For Experiment 2, it has 29 successes out of 40 attempts in predicting the direction of future trends.The accuracy rate of prediction shown in Table 5 has successfully warped the similarity across the patterns to predict the trends.The experiments have proven that the trend patterns repeat itself in different time stamp.Furthermore, we also proved that our approach, which used LRL, two-stage of MLP and DTW, can achieve a good prediction result.

CONCLUSION AND FUTURE WORK
The proposed approach LRL and segmentation rules in this paper has successfully analysed the different types of trend pattern -"Normal", "Skew Left" and "Skew Right" from "Uptrend" and "Downtrend" groups.Our experiments have shown that MLP can classify the different types of trend patterns, and stored the values of the resultant trend vector for DTW to predict the future exchange rate trends.In addition to this study, we discover that the trend patterns repeat itself in different time stamps.Moreover, the historical trend patterns are able to utilise as features to predict the future trends.The experiments also prove that LRL, two-stage of MLP, Kernelbased Tracing algorithm with DTW significantly improves performance of the prediction.
In the future work, the performance of DTW will be optimised to O(log (n)) in predicting the trend move.This is because the current time complexity of DTW takes O(n 2 ), which used brute force computation to compare the unknown partial patterns sequentially along with all the trend patterns from each cluster for predicting the trend movement.Furthermore, different Forex data would discover different type of trend patterns.It will be interesting to apply the proposed prediction model to predict the occurrence of these patterns.

Figure 1 .
Figure 1.Process Flow of Proposed Prediction Model.

Figure
Figure 5. MLP Learning Process
12unmatched information between the patterns.To identify the unmatched vector from the table, we introduced Kernal-based Tracing algorithm.

Figure 8 .
Figure 8. Financial Trend of AUD -USD in First Week of January 2012.

Figure 8 .
Figure 8. Financial Trend of AUD -USD in First Week of January 2012

Figure 9 .
Figure 9. Financial Trend of AUD -USD in Second Week of March 2012.

Figure 10 .
Figure 10.Financial Trend of EUR -USD in Third Week of October 2012. 14

Figure 8 .Figure 9 .
Figure 8. Financial Trend of AUD -USD in First Week of January 2012

Figure 11 .
Figure 11.Financial Trend of EUR -USD in First Week of December 2012.

Figure 10 .Figure 11 .
Figure 10.Financial Trend of EUR -USD in Third Week of October 2012

Figure 14 .
Figure 14.Prediction of EUR -USD Trend Movement on Week 2 January 2013.

Figure 15 .
Figure 15.Prediction of EUR -USD Trend Movement on Week 4 February 2013.

Figure 16 .
Figure 16.Prediction of AUD -USD Trend Movement on Week 1 January 2013.

Figure 17 .
Figure 17.Prediction of AUD -USD Trend Movement on Week 2 March 2013.

Table 1
Clustering Group Result using Proposed Features, Segmentation Rules and K-Mean Algorithm

Table 3 .
Result of First-Stage pf MLP Model for classifying different types of patterns in each group

Table 4 .
Classification Result of Identifying Trend Patterns

Table 5
Result of Predicting the Future of Trend Movement