1.1 Background

The inherent dangers of mining operations, particularly underground, are driving a growing demand for automation solutions that will remove personnel from hazardous and unhealthy environments. However, due to the relatively unstructured nature of the production environment and the attendant difficulties regarding accurate equipment monitoring, the mining domain represents a particularly challenging industry sector for the application of automation technology. Traditional closed-loop automation techniques drawn from the factory floor typically assume a high degree of certainty regarding key equipment parameters (such as position or status) that is often unavailable in the mining context. For this reason, probabilistic methods based on data analytics have shown more promise for managing and automating the mining process. This paper examines the use of such techniques in the context of a key mining automation challenge: the development of robust and accurate equipment localisation systems.

1.2 Problem description

While surface mining applications are now employing both RTK-GPS (Real Time Kinematic Global Positioning System—an enhanced version of standard GPS that employs a ground station to improve position accuracy) and extensive wireless communications systems to assist with accurate equipment localisation (Wei et al. 2014), underground operations such as longwall coal mining (Singh 1997) have no GPS access and typically very limited wireless access. A significant problem that must be solved in order to achieve continuous autonomous mining in longwall coal mines is the measurement of small changes in the position or orientation of specific sections of mining equipment over extended operational periods. For the case of longwall mining, two key parameters are longwall creep (cross-track position) and longwall retreat (along-track position) with respect to gate road travel. An outline of the longwall mining process is necessary in order to explain these problems in detail.

1.3 The longwall mining process

A coal seam is typically a contiguous wedge of coal, of varying height, sandwiched between layers of rock. The goal of the coal mining process is to extract as much of this seam as possible (or at least, the high-quality coal portions thereof) in an efficient and safe manner, with minimal impact on the surrounding environment. In a longwall coal mine two long, horizontal and permanent tunnels known as gate roads are cut into the coal seam to form the main boundaries for a large rectangular block of coal, known as a longwall panel. The gate roads are typically five to six metres in width. A typical panel is 200–300 m wide, and two to three kilometres in length (the length of the gate roads). At the end of the gate roads the major pieces of mining equipment are installed across the back of the panel, creating the longwall face (Fig. 1). At regular, surveyed intervals along the gate roads, cross-tunnels known as cut-throughs are established to provide access and assist with the management of mine ventilation (Singh 1997).

Plan view of a longwall coal mine

The key parameters to be measured and tracked are known as longwall retreat and longwall creep. Longwall retreat is simply the progression of the longwall face into the unmined coal panel as shown in Fig. 1. As the mining progresses, however, geological and equipment interactions creates variation in pressure across the longwall face, leading to incremental sideways slippage of the face equipment into the gate road, known as longwall creep (Fig. 2). This sideways movement is undesirable, and if left unchecked can lead to the longwall equipment eventually running up against the gate road rib, effectively bringing the face to a halt: a situation that is very difficult to resolve. Retreat and creep thus correspond to the along-track and across-track position of the mining equipment in the longwall gate road. Because this equipment is mechanically linked to the entire longwall face, accurate localisation of the gate road machinery enables improved management of the longwall process. Both relative localisation (i.e., measurement with respect to surrounding mine infrastructure such as tunnel walls) and absolute positioning within the overall mine region are critical.

Detail view of longwall face showing creep and retreat

1.4 Candidate sensors for position measurement

To achieve accurate localisation, a variety of sensing technologies have been considered in various mining contexts. For relative localisation (e.g., vehicle collision avoidance applications), active ranging technologies such as laser scanners and radar-based sensors are solutions that have been employed in similar non-mining applications. For absolute localisation, the absence in the underground of a pervasive positioning infrastructure such as GPS has led to the use of alternative technologies, in particular distributed sensor networks. For applications also requiring high accuracy, inertial navigation systems have been successfully applied by Australia’s CSIRO to machine guidance of both longwall and continuous miner applications (Ralston et al. 2014). Nevertheless, even high-end inertial systems are prone to drift with regard to the position estimate accuracy over time, and an external reference sensor is therefore required to place a bound on this source of error by correcting the inertial estimate on a periodic basis.

A scanning laser sensor has previously been employed to perform this position correction task in the context of longwall automation, however this is an expensive solution that requires a large flameproof cabinet to house the scanning laser for safe operation in the explosive environment of an underground coal mine. More significantly, laser scanner ranging performance is easily degraded by the presence of airborne or surface dust, which is a major limitation in a mining context. When water is present, as is always the case in an underground coal mine, the surface can rapidly become coated with an opaque layer of mud that completely obscures the instrument’s field of view (Fig. 3).

Surface mud coating a laser scanner employed in a mining operation

1.5 Radar-based measurement sensor

In response to these limitations, it has therefore been proposed that a radar-based sensor be trialled as a more suitable external reference measurement system, given the superior performance of radar in dusty environments. Previous published work has discussed the development of this radar sensor, and the initial results from a mine-site trial (Hargrave et al. 2014). The current paper examines the application of various probabilistic data analysis techniques to this raw radar data, to determine if sufficient information can be extracted to provide a useful monitor for longwall creep and retreat in a production context. Of the two measures, creep (the across-track distance to the gate road rib-wall) is both more straightforward (a direct application of radar ranging) and the more critical measurement, as it provides direct feedback of a potential collision between the equipment and the tunnel walls. The retreat measurement is more challenging, as it requires tracking of the along-track position of the longwall equipment based on indirect inference from the adjacent rib-wall, as there are no stable along-track targets in a typical gate road. The initial hypothesis for the study was that the ubiquitous roof and rib structural bolts, with their large plate washers, would make excellent candidates for radar tracking, as they present a strong radar target that would contrast well with the coal wall background (Fig. 4). A series of measurement campaigns, both synthetic in-house testing and actual underground trials, were therefore planned to test this hypothesis and attempt to resolve both the retreat and creep measurement challenges.

Rib wall with bolt-plates visible on the surface

The specific radar system developed for this application was adapted from an Endress-Hauser FMR 250 industrial radar level sensor (Hargrave et al. 2014). The primary consideration in selecting this sensor was its certification for use in explosive environments, which is a critical requirement for operation of electrical equipment in the methane-rich atmosphere of an underground coal mine. Further details regarding the radar system development can be found in (Hargrave et al. 2014).

2.1 Initial radar system testing

Initial radar testing was carried out at CSIRO Energy’s premises in Brisbane. The radar unit was mounted on a moveable gantry above a box filled with coal to allow measurements at various points across the coal pit. Testing included basic ranging to the coal surface, and also detailed analysis of the changes in the radar response based on the presence of various targets on the coal surface that would typically be encountered in the mine environment, e.g., the plate-washers (Hargrave et al. 2007).

The first field test of the radar sensor was carried out in a disused section of an underground colliery in New South Wales, Australia, and targeted a very small section of a tunnel rib wall, scanning only three consecutive bolt plates (Hargrave et al. 2014). The rib-wall face was otherwise clear of any clutter, which provided ideal conditions for initial proof-of-concept testing. For the full field trial of the sensor under realistic conditions, a significant section of mining roadway was necessary, under more realistic conditions including the presence of piping, signage and other metallic structures found in a typical coal mine.

2.2 Description of underground mine test site

Permission was obtained to use a pre-production gate road at an operational longwall mine to conduct the field trial. This gate road was under development for future production, and hence was as close to an operational environment as was possible given that the test platform contains numerous uncertified items of electrical equipment. The mine provided support to carry out this field trial, including examination of all the non-certified equipment, the necessary dispensations to use these items in the proposed test location, and the provision of personnel to supervise the trial.

The test site was a 50 m length of gate road, close to the high wall of the mine. The testing took place between two side-roads branching off from the main roadway tunnel, known as cut-throughs. This was an ideal test location as cut-throughs are a common feature of all roadways in an underground coal mine. Cut-throughs are a very obvious feature for detection, and their locations are accurately surveyed, hence they can be used as correction points for any cumulative localisation errors. Roadway development for the test location was complete at the time of the trial, including all the typical mine site infrastructure such as air-lines, hydraulics, signage, and electrical distribution systems.

2.3 Sensor platform system

The underground trial featured a comprehensive set of sensor hardware (Fig. 5), mounted on a test platform designed to set the instruments at a similar height to that of the gate road mine machinery. A series of customised software applications were developed to acquire and synchronise data from the various sensing instruments.

Sensor suite on the test platform, radar enclosure on the right

The sensors employed for the trial were:

1. (1)

   The primary radar unit, based on a 26 GHz Endress-Hauser instrument, as discussed in Sect. 1.

2. (2)

   Three SICK LMS 200 laser scanners, two mounted broadside to the direction of travel scanning both tunnel walls (to verify the creep measurements) and the third in the direction of travel (to provide a basis for comparison with the system retreat estimate).

3. (3)

   A video camera with low-lux capability for use in the underground.

4. (4)

   A data acquisition PC (standard laptop).

The real-time data acquisition system time-synchronised the data from the radar and the laser scanners, allowing rapid post-processing, comparative analysis and data fusion. The post-trial data processing was carried out in Matlab, however the data storage and retrieval protocols were developed for eventual integration into an existing real-time longwall automation software system.

2.4 Description of system testing

The field trial was carried out over two days, beginning with a series of measurements along a defined section of gate road at various offset distances from the rib wall. Targets were positioned at each end of the test course to provide a reliable along-track reference from the forward-looking laser scanner for the radar data. One feature of note with regard to the gate road was the presence of extensive mine services (water, power, and ventilation) located along one side of the gate road on the same side rib-wall as the cut-throughs (Fig. 6). The other (non-cut-through) rib of the tunnel was relatively clean (see Fig. 4). This contrast between the service-side and clean-side ribs is typical of underground mines, and provided a good opportunity to examine the radar sensor performance for both cases. Multiple separate test runs were carried out facing both sides and at different offset distances with the radar sensor facing perpendicular to each wall in turn. In total, 26 datasets were acquired over the two day test period.

Cluttered rib wall and cut-through

For the majority of the testing, the radar sensor was positioned at 90° to the along-track progress of the test trolley with the sensor pointed directly at the rib-wall to provide a direct measurement of the creep distance. The rib wall in this section of the mine was unmeshed (see Fig. 4), unlike many mines where a steel meshing, covering the entire rib surface, is secured to the walls using the bolt plates. This mesh is used to prevent the collapse of smaller sections of roof or rib that could cause injury to personnel. Because the metal mesh presents an additional clutter source for the radar sensor, additional scans were also made of the roof (which was meshed, Fig. 7) along the full trial length of the trial roadway. These additional scans were performed to ensure that a truly representative dataset could be generated of the most common roof and rib conditions encountered in the vast majority of underground coal mines.

Meshed roof in the trial roadway

2.5 Data analysis plan

To estimate across-track movement (creep) and along-track movement (retreat), the data analysis plan first sought methods to isolate these two independent parameters from the raw radar signal data. The initial processing for display of the raw data is presented in Sect. 3, while the techniques employed to extract the two parameters of interest are discussed in Sect. 4.

Both the laser and radar data were processed for initial display immediately following each trial run. The raw radar data consist of the “envelope curves” of the radar signal. Through the application of masking and background subtraction filtering, signal artefacts such as gain bias and cross-talk between transmit and receive circuitry were removed (Skolnik 2001). The individual radar traces were then stacked to develop a three-dimensional, time-varying display of the radar output. Figure 8 shows a typical result for a test run at a radar offset distance of four metres from the rib wall. Two notable features seen in this figure are: the clear gap associated with the side tunnel corresponding to a cut-through, and the frequent peaks in the main signal return corresponding to the highly reflective wall-mounted bolt-plates.

Reconstructed radar data for a section of gate road with cut-though

Both of these features (the cut-throughs and the higher amplitude returns from the bolt-plates) are valuable aids for measuring the along-track distance.

The output from the laser scanners was used to provide ground-truth measurements for comparison with the radar data. By merging data from the forward-facing scanner, a reconstructed plan view of the section of gate road evaluated (Fig. 9). It shows both of the cut-throughs that bounded the region used for testing. Note that the figure’s vertical axis is highly compressed in this reconstruction in order to represent the entire test site in a single image: the length of the test segment is approximately 50 m, while the width is only five metres. The deviation in the middle of the roadway section is therefore highly exaggerated in this figure, as the gate road appeared straight when viewed onsite. In addition, several tests were performed with the laser scanners oriented to capture the full surface of the gate road rib and roof (Fig. 10). These scans were performed in order to develop a more complete picture of the mine environment, and to provide an opportunity to trial some radar/laser data fusion techniques.

Laser-reconstructed 2D view of the test site showing cut-throughs: note that the vertical scale is compressed in order to show detail of the horizontal features on a single image

Laser-reconstructed 3D view of a section of gate road ribs and roof

As noted in Section IID, scans of the roof with both radar and laser sensors were also acquired in order to test the efficacy of the technology in the presence of a meshed surface (Fig. 7). A detailed section of the roof reconstructed from the laser scanner data is shown in Fig. 11. In both this image and Fig. 10, the colour map variation corresponds to the laser range.

Laser-reconstructed view of a detailed roof section with meshing

4.1 Initial radar processing and creep measurement estimation

First-pass processing of the raw radar sensor data involved the isolation of both the amplitude and position of the maximum signal return for each radar trace. This feature ideally corresponds to the reflection from the rib-wall and thus may be used to provide a creep estimate. It should also vary depending on the presence or absence of the reflective bolt-plates at that point along the gate road and thus provide a waypoint reference for retreat measurement. The stacked raw radar trace data, the range to the maximum echo point and the maximum amplitude of the return for each trace are shown in Fig. 12.

Raw radar data with range estimate and maximum echo amplitude

As is clear from a cursory inspection of Fig. 12, the raw maximum-echo radar data is quite noisy, and thus filtering is required in order to obtain a reasonable position estimate for creep and to attempt to mask any peaks corresponding to the bolt-plate waypoints. For the creep estimate, both a Moving Average (MA) filter and a Kalman Filter (KF) were applied to the maximum echo radar range-to-wall estimate to smooth out discontinuities (Fig. 13).

Creep position estimate, comparison of moving average and Kalman filters based on the filtered versions of the raw radar signal

As expected, a well-tuned KF provided a better estimate of the creep distance than the MA filter, reacting to rapid changes in the range more smoothly than the MA filter could do for a reasonable window size. The MA filter result could be made arbitrarily smooth by selecting a large enough window, but this introduces an unacceptable lag in the position estimate. The KF also provided a closer match to the creep estimate obtained from the laser scanner data when the two sensors were compared.

The creep position estimates obtained from the KF were compared to data from the laser scanner that was directed towards the same rib wall as the radar sensor. The laser scanner generated a 2D scan line of discrete points (rather than a single point measurement as is the case for the radar), so the comparison was performed using the mean value of the central 10° fan of laser data points. The laser data were otherwise unprocessed for the creep estimation task, and were synchronised with the radar traces based on the common system clock on the data acquisition PC.

For validation, a comparison of the typical system creep performance is shown in Fig. 14 for a 50 m run of the trolley at a nominal offset from the mine rib wall of 1 m. The radar sensor, when combined with a well-tuned filter, showed excellent correlation with the laser scanner data; its performance exceeded the accuracy and precision requirements for the basic creep measurement application. This result was found to be quite consistent across the many datasets generated in the field trial.

Radar creep measurement compared with laser scanner data in close agreement

Despite the presence of mesh on the roof, similar results were also obtained for gate road height measurement as extracted from the raw roof scanning data (Fig. 11). A typical result for the radar versus laser tracking for the roof scenario is shown in Fig. 15, where the correlation between the two sensors is once again excellent.

Radar roof height measurement compared with laser scanner data

4.2 Extended trial of gate road height measurement

An application of the creep measurement technique was later applied on an actual longwall gate road (rather than a main roadway) at an underground mine in Queensland, with the goal of measuring the gate road height (i.e., sensing the range to the roof, rather than the rib). The length of gate road measured during this operational trial was over five hundred metres long (involving a trial period of almost two months), and both radar and laser-scanner data were acquired for validation purposes, as shown in Fig. 16 (results are shown for the first 10000 samples, or approximately 500 m). The correlation between the radar and laser data was quite strong for the majority of the roadway, with the main deviations being due to the effect of dust on the laser scanner system. The large vertical deviations in the blue laser data shown in Fig. 16 occurred when the window of the laser scanner flameproof became obscured by dust or mud, leading to inaccurate estimates for that sensor until the window could be cleaned. This result demonstrates the value of the radar-based sensing approach, as the radar was installed in an enclosure adjacent to the laser scanner yet remained unaffected by the same dust conditions that effectively blinded the laser. Smaller deviations occurred where the laser scanner result (which is obtained by averaging the height of all samples across an extended roof section) was affected by local pockets in the roof that led to an overestimate of the roof height, which again highlights the advantage of the radar for this application. This extended trial also demonstrated that the core radar technology is quite robust for use in practical mining environment.

Radar (red) versus laser (blue) data for a second, extended, longwall gate road height measurement trial

4.3 Retreat measurement estimate

The working hypothesis for a retreat measurement system was based on discriminating the highly reflective rib-wall bolt-plates from the background coal, and using these features as reference waypoints for a localisation solution. Bolting patterns are strictly controlled in underground mines, as they constitute the primary support preventing gate road collapse. The number of bolts installed in a segment of gate road (e.g., between two cut-throughs) is therefore a known environmental mining parameter, and by tracking the number of bolts encountered since the previous cut-through the along-track position of the mining machine can be accurately estimated.

Figure 17 shows the raw maximum echo output for the side-facing radar against the service-side rib-wall, together with the MA filter and KF results. The superiority of the KF was again notable, as the responsiveness of this filter tracks the rapid change between the bolt-plates and the background coal more closely, improving the discrimination result. In both filtered datasets, the consecutive peaks and troughs associated with many of the passing bolts are easily discerned.

Maximum amplitude echo points with KF and MA filters

Although an initial examination of the maximum radar return data in Fig. 17 appears encouraging, closer inspection reveals that the number of peaks do not correspond to the number of bolts present (40 bolts) in the tested segment of the gate road rib. While some bolt-plates provided a strong reflection and hence maximum signal amplitude, in other cases the contrast with the background coal was not sufficient to allow discrimination. This appears counter-intuitive given the strong scattering from a plate structure; the explanation lies in the fact that plates are highly specular reflectors, so that the strong reflection from the target will scatter in a narrowly focussed direction corresponding with Snell’s law. If the plane of the plate is mounted at 90° to the incident wavefront it will provide a strong radar target, with most of the signal reflecting back to the radar receiver. However, if the plate is angled a few degrees off-axis then most of the signal energy will reflect away from the sensor and the overall return energy may be quite low. In addition, there are other significant scattering sources present (particularly on the service-side of the gate road) that will constitute unwanted radar “clutter” that further complicate the detection and measurement process.

Given the feature ambiguity associated with a technique based purely on maximum return amplitude, it was clear that a classification engine would require a richer feature set from the raw radar data in order to discriminate successfully between the bolt-plates and other structures, or even the coal wall itself. Identifying strongly correlating features for the bolt-plates therefore became an important task for the research.

4.4 Visual analytics for feature selection

Because of the complexity of the detection environment (the irregular surface of the rib walls, the presence of clutter sources, and the significant variation in the installed angle of the bolt-plates) recognising consistent features for the target was not a trivial task. In order to aid this process, a novel data fusion and visual analytics approach was employed based on combining the laser scanner and radar data in a 3D interactive display to facilitate examination of significant features (May et al. 2010). The goal of this work was to provide the researchers with a comparative tool whereby the raw radar data could be examined directly with the surfaces generated from the laser scanner data. By visual inspection of the data, it was hypothesised that different features in the radar data might be identified that could be useful for discrimination purposes.

Using the trial runs in which the laser units were oriented to scan in a vertical plane in order to capture the full surface of the gate road ribs and roof (as shown in Fig. 10), and exploiting the synchronisation between the laser and radar sensors, the radar return information could be overlaid against the high-resolution point-cloud structures from the laser data. The resulting 3D surfaces allowed the radar data to be examined in a highly detailed alignment with the physical surface structures as revealed by the laser scanner (shown conceptually in Fig. 18). By adjusting the section of the radar return overlaid on the projected rib wall structure, different aspects of the radar return could be interrogated for changes and associated with specific features in the laser point cloud, in particular the bolt-plates.

Visual analytics concept showing mine structure based on laser point cloud overlaid with radar data

This method of data fusion for visual inspection was applied to the combined laser and radar datasets from the mine site trial, allowing the radar return information to be projected onto the 3D surface generated by the laser scanner data. Figure 19 shows the resulting combined information for the section of roof shown in Fig. 10, with the laser-generated surface now colour-mapped to the corresponding radar return amplitude data. In Fig. 19 the strong reflections from various radar targets are clearly visible, and their correspondence to particular features of the laser data such as the bolt-plates and other infrastructure elements can be readily identified. The combination of radar and laser data provides a powerful mechanism for characterising the particular radar dataset features that correspond to the bolt-plate targets, which are the primary features of interest for establishing a way-point tracker for retreat measurement.

3D roof surface with overlaid radar amplitude

This visualisation process proved an effective way to reveal useful features beyond the initial strong echo. One important example was verification of the so-called “double bounce” effect which occurs when a secondary echo is observed for a given target at twice the true target range, due to a round-trip reflection between the radar sensor and the strongly reflective target (the phenomenon can be observed in the stacked raw radar traces shown in Fig. 12) (Skolnik 2001). Although not observed for every target, this phenomenon was clearly characteristic of the bolt-plates, and was not observed even in the strong scattering from clutter sources. Where present, this significant additional feature provided a useful supplementary source of information regarding the observation of a bolt-plate, and thus was incorporated into the classifier algorithm to improve discrimination. Other features such as characteristic variations in radar reflection intensity across the face of a bolt-plate are under investigation.

4.5 Exploration of feature generation for visual analytics

Further improvements for estimating the position of the bolt plates could be realised by incorporating different feature sets into the visual analytics approach. For example, ambiguities associated with the class of target identification methods based on signal strength alone could be significantly mitigated through the use of resonant frequency target identification techniques (Hargrave et al. 2013), which have remarkable properties for aspect-independent imaging. The features extracted using this technique could be incorporated into the classifier algorithm to further refine the discrimination results.

A variety of classifiers have been trialled for use with the radar data. A decision tree structure, which allows a simple implementation of rules (e.g., if the maximum amplitude of the return is within a given range, classify the target as a bolt-plate), has shown promise as a suitable algorithm for the application (Fig. 20). This technique has the distinct advantage that any a priori information regarding the structural hierarchy of the gate road configuration and/or equipment can be meaningfully incorporated to improve the detection and estimation accuracy.

Initial decision tree for classifying bolt-plates, coal and clutter

We have described a novel radar-based system for underground mine machine localisation. The approach avoids the need to install additional reference infrastructure by exploiting mining structures already present in the underground mining environment, such as bolt plates and cut-throughs, to generate feature-rich radar signal returns. Different probabilistic data processing techniques were explored in order to provide estimates of the equipment’s along-track and across-track position. These estimates were subsequently validated through the use of laser range scanners and found to be in close agreement. The success of these trials has spurred development of a modified system that is now ready for integration into an operational mine machine automation system. Work is ongoing to improve the classifier algorithm performance with the view to incorporating these results into an operational automation system.