A NEW APPROACH TO HIGHWAY LANE DETECTION BY USING HOUGH TRANSFORM TECHNIQUE

This paper presents the development of a road lane detection algorithm using image processing techniques. This algorithm is developed based on dynamic videos, which are recorded using on-board cameras installed in vehicles for Malaysian highway conditions. The recorded videos are dynamic scenes of the background and the foreground, in which the detection of the objects, presence on the road area such as vehicles and road signs are more challenging caused by interference from background elements such as buildings, trees, road dividers and other related elements or objects. Thus, this algorithm aims to detect the road lanes for three significant parameter operations; vanishing point detection, road width measurements, and Region of Interest (ROI) of the road area, for detection purposes. The techniques used in the algorithm are image enhancement and edges extraction by Sobel filter, and the main technique for lane detection is a Hough Transform. The performance of the algorithm is tested and validated by using three videos of highway scenes in Malaysia with normal weather conditions, raining and a night-time scene, and an additional scene of a sunny rural road area. The video frame rate is 30fps with dimensions of 720p (1280x720) HD pixels. In the final achievement analysis, the test result shows a true positive rate, a TP lane detection


INTRODUCTION
Research in Intelligent Transport System (ITS) can be divided in to several categories.Some of the categories are based on image computational processing such as vehicle detection, road lane marks detection, and road obstacle detection.In this paper, we proposed a road lane detection system for three application implementations, which were road triangle extraction using vanishing point, road width calculation for scale factor determination and Region of Interest (ROI) for object detection area.
In image processing techniques for road area scenes, many parameters can be obtained from image frames such as the number of vehicles, turning traffic flows at intersections, and speed measurement (Chong et al., 2013).Generally, there are two types of videos that are implemented in their studies; static and dynamic.The static represents the position and location of the camera such as CCTV, which records and monitors the road scenes at certain positions.Meanwhile, in the dynamic approach, the camera location is not permanently located but it moves together with the targeted objects and surrounding objects, such as the on-board camera which records the road environment in the moving vehicle.The main difference between both approaches is the background motion of the recorded scenes.Static video is composed of the background that is almost static, while in the dynamic video, the background and foreground are almost in the same motion, causing difficulty to distinguish between background and foreground.
In static video approaches, since the background in the road scene is almost static, the background subtraction method is widely used to detect the moving object within the foreground and classify the background model as non-interested object that is not in motion (Brutzer, Höferlin, & Heidemann, 2011;Haines & Xiang, 2014;Leon & Hirata Jr, 2012;S. Li, Yu, Zhang, Yang, & Bin, 2014;Sirikuntamat, Satoh, & Chalidabhongse, 2015;J. Wang, Lu, Gu, Zhou, & Chai, 2014).However, this method faces difficulties with outdoor scenes, and it is an unsuitable approach for dynamic video scenes due to differences in focus scenes.
Another method to segment the background and foreground is by motion characteristic differentiation.The motion flow spreads out gradually at the side of the image and the velocity will increase as the scene moves closer.This information is used to classify the background model, ROI and also the foreground (Jazayeri, Cai, Zheng, & Tuceryan, 2011).The knowledge of motion is separated into different classes to distinguish the object of interest with the uninterested object.This approach needs more computational time which increases the processing time.The vehicular autonomous system is important in autonomous vehicle guidance to avoid collision with another vehicle.It avoids the road obstacle by providing warning to the driver in the case of a dangerous situation.This approach uses vehicular on-board units that detect the surrounding vehicles, motion status and updates the status to the driver (C.Wu, Peng, Huang, Zhong, & Chu, 2014).
In the studies of dynamic background removal by Brutzer et al. 2011), the details of the basic sequences method have the capability to represent each background model by classifying the background into an uninteresting movement, thus, classifying it as an invalid object for detection process.Moreover, Zhang, Wu, & bing Yin (2010) introduced an adaptive background update to differentiale the motion property of the dynamic background on road scenes, where it uses the lighting changes of the scene and moving pixel properties.Based on lighting changes, illumination is important depending on the weather in the road scenes which makes it disadvantageous during night time.
On the other hand, the most significant method used in lane detection development is Hough transform (Ghazali, Xiao, & Ma, 2012;Hassanein, Mohammad, Sameer, & Ragab, 2015;Mao & Xie, 2012).The Hough transform operator detects straight lines and curves.It is the most reliable method to detect straight lines by counting each unique equation for every possible line that can be detected from each point in the image (Mariut, Fosalau, & Petrisor, 2012).
To develop a boundary between road and non-road areas, there are multiple available methods that can be employed.Image processing techniques such as image pre-processing, line detection, image masking and image multiplication can be implemented for background elimination, deletion and image segmentation.Road area extraction has been utilized in many researches to segment the road and the outer road.Li et al. combine structure information and classification of texture from the road scenes with super pixel scene segmentation (J.Li, Jin, Fei, & Ma, 2014).The studies of P.-C.Wu, Chang, & Lin (2014) state that the color of lane marks are usually white and yellow and some are usually painted red; displaying a high contrast compared to the road surface color.Using that color information, they enhanced the lane character of the road surface by increasing the lane-mark intensity values using Equation 1. (1) Vanishing point is usually located in the middle of the image and the road region is below that point.Therefore, detection of the vanishing point is one of the important parts in determining the road area region.This distinguishes between road and non-road Ding & Li (2015) found that the straight lines in the road image extract the road region for vanishing point support.The road region is segmented based on the vertical lines.There are vertical lines that represent both sides of the road such as buildings, trees, fences and other vertical objects, then this segmentation is based on the enveloping of the vertical lines while the vanishing point is based by selecting straight lines that are suitable for a candidate and calculated by using the mean shift clustering.On paper (Haines & Xiang, 2014), the method divides the image plane into the interior region (IR) and the exterior region (ER), to represent the image region, and the three vanishing points are detected sequentially by using the 1D histogram.This method, however requires much more computational time, thus, making it longer to process.
In terms of processing time, the purpose of informing the region of interest in the detection area is to reduce the detection computational time.It is claimed by Baek, Han, Kang, Chung, & Lee, (2016), the processing time is decreased when detection and verification are processed within the predicted area (ROI) instead of scanning the whole image.The detection integrity and accuracy can be earned when the processing area is limited to its purpose area (S.Wang, Liu, Gan, & Cui, 2016).Moreover, with the utilization of high definition videos in algorithm, the processing time is increased ultimately with the higher quality of the video dimensions.Therefore, a pre-processing technique is needed to reduce the noise and focus on the designated area which consequently reduces the computational and execution time.
In summary, this paper applied the Hough transform for lane detections.In this research, we assumed that the road was flat, thus the road lane was straight, and a triangle was coming out in the output of the road (Chong et al., 2013).Therefore, to find the vanishing point of the road rk intensity values using Equation 1.
B line, as described in Equation 2.
point is then obtained by applying Equation 5.
triangle, lane detection was used to detect lane marks on the road.Using a simple mathematic formula, the intersection point between the lanes detected on the left side and the right side road mark, was used with the vanishing point.This vanishing point is to extract road triangle, calculate road width and building ROI for the detection area.The next section demonstrates the lane detection system using our proposed algorithm.
Validation and tests are proven in the final experimental result using multiple video footage presented to substantiate the effectiveness of the proposed method.

ALGORITHM DEVELOPMENT
The overall algorithm development of the background elimination system can be viewed in the flow chart of the system as shown in Figure 1.The system starts by reading the input image frame sequence of the road video that was produced by the on-board camera.Next, comes the image pre-processing operations, for image enhancement to highlight the road lane features.Image enhancement is carried out then, using image smoothing techniques as a noise filter, and for the edge extraction, the Sobel operator is utilized.For lane detection, the Hough transform technique is used to detect the left most and the right most sides of the lane, and a reference line is drawn into the image frame.Three intersection points between the line are obtained by using a point of intersection between two lines, A and B line, as described in Equation 2. (2) From the intersection point, three main operation parameters can be obtained.
The first one is the center point which is known as the vanishing point that is located at the end of the road.Then, the road width, which is calculated using the Euclidean distance formula.Lastly the Region of Interest (ROI) formed is based on the left and right lane intersection point to distinguish the road area and the non-road area.

Image Pre-processing
Image pre-processing is an important operation in improving the image and enhances the road lane marks in a sequence of image frames.At this point, the image frames are processed in the lower area where the road lanes lie, excluding the horizon area to avoid misdetection.Then, the image frame is converted to greyscale format.This is because luminance is important in distinguishing visual features.The color image information does not help in identifying important edges in the image, and the complexity of the codes takes as much as three times longer for processing time in each pixel.Hence, image conversion is important in this process by using the thresholding value.This value saturates the values of the image pixel to be between 0 (black) and 1 (white) to convert the greyscale image into a binary image.
The blurring technique, known as Gaussian smoothing, is used in this algorithm to smooth the image to removes unwanted features and small noises present in the image before applying the edge detection operator.Figure 2 and Figure 3 show the before and after of applying the blurring techniques.
From the figures, it can be noticed that the left road divider disappears after blurring is applied.This is a crucial image improvement because the road divider interferes on lane detection operations.
e, as described in Equation 2.

𝐴𝐴 ∩ 𝐵𝐵
(2) t is then obtained by applying Equation 5.After the blurring techniques, median filtering is used to further improve image enhancement.Median filter is a nonlinear method to remove the noise from an image that is commonly used in the imaging process.The advantage of using this filter in lane detection at this point of time is that it preserves the edges although the noise removal is applied (Fan, Wang, Cayre, & Xiong, 2015).Thus, it is an effective operator in this system for detecting the edges for the next operation to keep its important features.
The Sobel operator is used to extract the edge detection on the video sequence.
The operator computations are shown in Equation 3: (3) The two (3x3) kernel of G x and G y are the directions of the image which denotes the vertical and horizontal derivatives.To yield the horizontal edge feature detection, the G y kernel from (2) is used to extract the horizontal edge pixels since lane features are more distinctive to horizontal features.
This operator satisfies the edge line criteria needed by the Hough transform for lane detection.Moreover, it has a low error rate of detection as it has an accurate edge detection from the image frame.The edge extraction is shown in Figure 4.
the lane-mark intensity values using Equation 1.
lines, A and B line, as described in Equation 2.
After the blurring techniques, median filtering is used to further improve image

Lane Detection
In the algorithm, the Hough transform is applied to detect two sets of solid lanes (left and right).The transform extracts the features that are used in estimating the lane parameters (Gaikwad & Lokhande, 2015) .Hough transform uses a parametric for a representation of a line to detect the lines in an image and is calculated as follows in Equation 4. (4) The variable ρ is the distance between the origin and the line along its vector perpendicular to the line, while ϴ is the angle between the x-axis and the vector.Thus, Hough transform can generate a parameter space matrix that corresponds to ρ and ϴ values in rows and columns respectively (Ebrahimpour, Rasoolinezhad, Hajiabolhasani, & Ebrahimi, 2012).
Figure 5 shows that the lane is detected and the lanes are marked using colored lines, the left lane is in green color while right lane is a blue line.These two lanes are the solid white lanes that are near to the non-road area for the left and the right sides.ne-mark intensity values using Equation 1.
A and B line, as described in Equation 2.
ane Detection the algorithm, the Hough transform is applied to detect two sets of solid lanes (left and right).
he transform extracts the features that are used in estimating the lane parameters (Gaikwad & okhande, 2015) .Hough transform uses a parametric for a representation of a line to detect the nes in an image and is calculated as follows in Equation 4. (4) The variable ρ is the distance between the origin and the line along its vector erpendicular to the line, while  is the angle between the x-axis and the vector.Thus, Hough ansform can generate a parameter space matrix that corresponds to ρ and  values in rows and lumns respectively (Ebrahimpour, Rasoolinezhad, Hajiabolhasani, & Ebrahimi, 2012).
Figure 5 shows that the lane is detected and the lanes are marked using colored lines, the ft lane is in green color while the right lane is a blue line.These two lanes are the solid white nes that are near to the non-road area for the left and the right sides.
Both green and blue lines mark the detected lanes across the image frames.Since the road flat, the lines are in line with the road lanes that intersect at the peak of the image.The point intersection at the road end points at the vanishing point.
Both lines are linear with slopes due to the straight lines being angled away from the rizontal as shown in Equation 5with two parameters, slope ( m ) and y-intercept ( b ), and the tersection point is then obtained by applying Equation 5.
Both green and blue lines mark the detected lanes across the image frames.
Since the road is flat, the lines are in line with the road lanes that intersect at the peak of the image.The point of intersection at the road end points at the vanishing point.
Both lines are linear with slopes due to the straight lines being angled away from the horizontal as shown in Equation 5with two parameters, slope (m) and y-intercept (b), and the intersection point is then obtained by applying Equation 5. (5) The point of intersection is the intersection between the two lines that overlap each other at the end of the road.Thus, the coordinate for the intersection is obtained and marked using the red circle as shown in Figure 6.A horizontal reference line is drawn in Figure 7 to find the intersection of the two lines on the left and right.Both lines will intersect with the reference line and result in a total of three points of intersection.These three points look like a triangle as shown in Figure 7 and this triangle area is defined as the Region of Interest (ROI).The ROI can be used as the focused region for vehicle detection and the classification algorithm.

𝐴𝐴 ∩ 𝐵𝐵
(2) n point is then obtained by applying Equation 5.The point of intersection is the intersection between the two lines that overlap each othe at the end of the road.Thus, the coordinate for the intersection is obtained and marked using th red circle as shown in Figure 6.
A horizontal reference line is drawn in Figure 7 to find the intersection of the two line on the left and right.Both lines will intersect with the reference line and result in a total of thre points of intersection.These three points look like a triangle as shown in Figure 7 and thi triangle area is defined as the Region of Interest (ROI).The ROI can be used as the focuse region for vehicle detection and the classification algorithm

Vanishing Point for Road Triangle Extractions
The three coordinates which are obtained from the intersection are used to create binary masking.This marking is used to highlight the ROI inside the triangle so that the unwanted region can be removed.The triangle mask is set to '1' and the outside to '0'in order to allow image multiplication techniques (Ibrahim, Ali, & Y., 2016) to be applied to the image with the mask created as shown in Figure 8.Since the outside of the road triangle is set to '0' value, all the objects in that region are completely removed in the result.

Road Width Calculations
Intersection value by lane detection with reference lines can be manipulated and used to calculate road width value by pixel.The distance is calculated by using Equation 6 that is derived from the Euclidean distance formula in two dimensions represented by P1 , (P1 1 , P1 2 ) for the left lane intersections and P2 , (P2 1 , P2 2 ) for the right lane intersections. (6) The distance value from the calculation is obtained in pixel value.Since the actual size of the two road lanes in Malaysian highways is estimated to be 3.5 meters for each lane, the total width of the two lanes is 7 meters.Using Equation 7, the scale factor to convert distance value in pixel to the real world can be computed.
lines, A and B line, as described in Equation 2.

𝐴𝐴 ∩ 𝐵𝐵
(2) intersection point is then obtained by applying Equation 5. the lane-mark intensity values using Equation 1.
lines, A and B line, as described in Equation 2.

Region of Interest (ROI) of Road Area
The extraction of ROI for road area for detection application such as vehicle detection is executed using the left and right lane intersection value.A bounding box that scans through the road area is formed using the two points P1(P1 1 , P1 2 ) for the left lanes and P2(P2 1 , P2 2 ) for the right lanes.Since the points are road lane values, the inner part of the road boundaries is focused and ROI is created in the other areas.

RESULTS AND DISCUSSION
To evaluate and test the performance of the algorithm developed, the video footages total 2366 frame and five scenes from the highways around Malaysia under normal driving conditions.Video frames with a frame rate of 30fps and dimensions of 720p (1280x720) HD pixels are used.These video footages are taken using the normal on-board dash camera installed in a car.The test and validation of the developed road lane detection uses four different videos of road scenes.Scenes are based on the weather conditions and the area such as rural area and highways.In the highway road scene, the weather conditions were normal and raining and during the scene was night-time while the rural area was taken on sunny days.

RESULTS AND DISCUSSION
To evaluate and test the performance of the algorithm developed, the video footages total 236 frame and five scenes from the around Malaysia under normal driving conditions Video frames with a frame rate of 30fps and dimensions of 720p (1280x720) HD pixels ar used.These video footages are taken using the normal on-board dash camera installed in a car.converting it to scale factor for real world measurement coefficient from pixel.
Table 2 shows the result from the use of 5 different videos for scale factor calculations.
Table 2 shows the results of the scale factor value calculated using five different video inputs of road highway scenes that consist of two lanes.Calculations are done using Equation 7 in the algorithm development section.The first 20 frames of a video with the lane detection system were used to calculate the lane width between the right most solid lanes and the left most solid lanes.Scale factor was used to convert the real-world distance from image pixel.Variation in the scale factor value happens when there are slight changes of camera angle and positions.Therefore, the placement of the camera is main effect factor for scale measurement.It is useful for robustness of measurement since the camera can change positions based on the type of vehicle used.
The implementation of the Region of Interest (ROI) detection area uses two intersection points unlike the vanishing point that requires three points.The operation of the execution is the same as the vanishing point and the results are shown in Figure 11.The detection area box developed follows the road lane to 1 and road fly-overs effect the true detection rate.Meanwhile, false positive occurances are due t road dividers and edges that are similar to road lanes making the detection of lanes to falsel detects the road dividers too.
Figure 10 shows the lane detection and lane-width measurement obtained from th Euclidean distance calculated from two points of intersection and converting it to scale factor fo real world measurement coefficient from pixel.Table 2 shows the result from the use of different videos for scale factor calculations.
Table 2 shows the results of the scale factor value calculated using five different vide inputs of road highway scenes that consist of two lanes.Calculations are done using Equation in the algorithm development section.The first 20 frames of a video with the lane detectio system were used to calculate the lane width between the right most solid lanes and the left mos solid lanes.Scale factor was used to convert the real-world distance from image pixel.
Figure 10.Road-width calculation focus on the road region area for detection.It can take the place of any object's presence in that area.The analysis results are based on the recognition rate of the vanishing point application with a 82.59 % lane recognition rate, since it fully depends on road lane detection as much as vanishing point detection.

CONCLUSION
In this paper, an algorithm for road lane detection was proposed, based on dynamic videos that were recorded by the on-board camera installed inside a moving car.The method applied in the algorithm was a combination of lane detection, vanishing point and image subtraction.Each component depends on each other, and if one did not work, the whole system would fail to give a removed background result.The method succeeded in the implementation of road triangle extraction, road-width calculation for scale factor determination and Region of Interest (ROI) for road boundary object detection (vehicle).
During lane detection, when then vehicle was moving the on-board camera covered the lane mark of the roads.Hough transform was still able to track the loss of lane marks by assuming the lane was still there by counting the number of lost frames.If the lost track was more than the defined number of frames, then it stopped the tracking operation.However, the tracked lanes cut across the vehicle during the tracking, thus, when masking was applied and the image multiplication was executed, part of the vehicle was lost.For that reason, an improvement needs to be done, especially in the lane detection part.However, the vehicle detection operation still can be implemented without any effect from the background components which can be achieved by using edge detection that uses a Sobel operator and a bounding box detector.

Figure 1 .
Figure 1.The overall flow chart of the system.

Figure 5 .
Figure 5. Left and right lane detection using Hough transform.

Figure 6 .
Figure 6.Point of intersection in circles.

Figure 7 .
Figure 7. Point of intersection with reference line.

Figure 7 .
Figure 7. Point of intersection with reference line.

Figure 7 .
Figure 7. Point of intersection with reference line.

Figure 6 .
Figure 6.Point of intersection in red circles.
14 e objects in that region are completely removed in the result.oad Width Calculations tersection value by lane detection with reference lines can be manipulated and used to calculate ad width value by pixel.The distance is calculated by using Equation 6 that is derived from the uclidean distance formula in two dimensions represented by for the left lane tersections and for the right lane intersections.igure 8. Road triangle masking.

Figure 9
Figure 9 shows the final image result of the vanishing point application implementations.Road triangles are formed from the intersection point of the road lane detected and the vanishing point.Image multiplication operation is carried out between the triangle image mask and the original image.The final output from the image multiplication produces a road triangle area eliminating the outside boundary of the road area.

Figure 9
Figure9shows the final image result of the vanishing point application implementations

Table 2
Scale Factor