Day 75: Cross‑Line Analysis & Statistical Cleaning – Masterpiece Edition | River Warrior

📉 DAY 75: CROSS‑LINE ANALYSIS & STATISTICAL CLEANING

⏱️ Estimated Reading Time: 16 Minutes | 🎓 Level: Professional Hydrographer / QC Analyst

Refining Bathymetry – Advanced Cross‑Line QC and Robust Outlier Rejection

Instructor: Engr. Rokib Hossain | River Warrior Academy

🏠 Course Homepage

📖 Table of Contents (Serialised)

Why Advanced Cross‑Line Analysis Is Non‑Negotiable
Cross‑Line Basics: Mean, Std, and IHO Limits
Statistical Cleaning: 3‑Sigma, MAD, CUBE, and Adaptive Filtering
Interactive Cross‑Line & Outlier Simulator
Integrated Workflow: Clean → Cross‑Line → Re‑Clean → Validate
Residual Error Analysis (Spatial Patterns, Systematic Bias)
Case Study: Bay of Bengal Cross‑Line Mismatch Investigation
Cross‑Line & Statistical Cleaning Checklist
Software & Tools
Frequently Asked Questions
Action Items & Next Steps

1. Why Advanced Cross‑Line Analysis Is Non‑Negotiable

Cross‑line analysis is the most powerful QC tool in hydrography. It compares depths from main survey lines with perpendicular cross‑lines at intersection points. The residuals reveal systematic errors (bias) and random noise. Day 47 introduced the basics; this day dives into statistical cleaning techniques that use cross‑line residuals to identify and reject outliers, and to fine‑tune the data until the residuals meet IHO standards.

A clean survey should have:

Mean difference < 0.1 m (Order 1a shallow).
Standard deviation of differences < 0.15 m.
No spatial trends (e.g., difference increasing with depth or across the survey).

🧠 Golden Rule: Run cross‑line analysis after every major cleaning step. If the standard deviation does not improve, you are probably removing real features, not noise.

🌊 River Warrior Pro-Tip: Iterative Cleaning

In the Bay of Bengal, we applied a 3‑sigma filter to the cross‑line differences themselves – any main line that consistently deviated was reprocessed. This iterative approach reduced the final cross‑line std from 0.22 m to 0.09 m.

2. Cross‑Line Basics: Mean, Std, and IHO Limits

.htmlOrder 1b (shallow).htmlDeep water (scale)那样0.5% depth那样0.5% depth那样> 30 m

IHO Order	Max mean diff (m)	Max std diff (m)	Depth range
Order 1a (shallow)	0.1	0.15那樣< 30 m
0.2	0.3那样< 30 m

Cross‑line differences are computed at intersection points. For MBES, the software extracts depths from both lines using bilinear interpolation. The resulting residual = depth(main) – depth(cross).

3. Statistical Cleaning: 3‑Sigma, MAD, CUBE, and Adaptive Filtering

3‑sigma (standard deviation): Remove points where depth deviates more than 3 standard deviations from the local mean. Simple, but sensitive to extreme outliers.
Median Absolute Deviation (MAD): More robust. MAD = median(|xi – median(x)|). Reject if |xi – median| > 3 × MAD.
CUBE (Combined Uncertainty and Bathymetry Estimator): Uses uncertainty propagation to weight soundings. Flags outliers with low weight.
Adaptive filter (slope‑based): Rejects points where the depth change between adjacent soundings exceeds a threshold (e.g., 20% of depth).

For cross‑line cleaning, apply the same statistical filter to the difference dataset to identify problematic intersections.

💡 In Qimera, use “Statistical Filter” with a 3‑sigma threshold after cross‑line analysis to highlight lines with abnormal residuals.

📊 Cross‑Line & Outlier Simulator

Simulate cross‑line differences and apply a 3‑sigma filter to detect outliers:

Number of intersections: Mean difference (m): Standard deviation (m): Add outlier fraction (0‑0.2):

Before cleaning: mean=0.02, std=0.14 | After 3‑sigma: 48 points kept, std=0.09

Outliers beyond 3σ are flagged; you would re‑check the associated lines.

4. Integrated Workflow: Clean → Cross‑Line → Re‑Clean → Validate

1️⃣ Initial cleaning (swath editor)

2️⃣ Run cross‑line analysis (full survey)

3️⃣ Identify lines with high residuals (e.g., >3σ)

4️⃣ Re‑clean those lines (or re‑process with new SVP)

5️⃣ Repeat cross‑line analysis

6️⃣ Compute final statistics and TPU validation

📈 A well‑cleaned survey may require 2‑3 iterations. Do not over‑clean – if you reject >5% of soundings, your acquisition parameters are likely wrong.

5. Residual Error Analysis (Spatial Patterns, Systematic Bias)

Plotting cross‑line residuals in a colour‑coded map reveals patterns:

Systematic bias (all residuals positive): Tide datum error or GNSS vertical shift.
Residuals increasing with depth: SVP error (outer beams smile/frown).
Residuals varying with direction (heading): Yaw or pitch misalignment.
Clusters of large residuals: Localised noise (bubbles, fish, electrical interference).

🔍 In Qimera, export the cross‑line differences as a shapefile and visualise in QGIS. Colour by residual value – patterns become immediately obvious.

6. Case Study: Bay of Bengal Cross‑Line Mismatch Investigation (2026)

Situation: After initial cleaning, cross‑line standard deviation was 0.23 m (Order 1a limit 0.15 m).

Analysis: Plotted residuals vs depth. Saw a trend: residuals increased from 0.05 m at 10 m to 0.35 m at 30 m.
Root cause: SVP cast was 6 hours old; a shallow thermocline had changed. Fresh SVP reduced residuals to 0.14 m std.
Iterative step: After applying new SVP, residuals still had a small bias (mean = 0.07 m). Adjusted tide datum by 0.05 m – bias disappeared.
Final result: Mean = 0.01 m, std = 0.12 m – passed Order 1a.

📊 The cross‑line residual map was included in the final report, demonstrating due diligence.

7. Cross‑Line & Statistical Cleaning Checklist

Perform initial cleaning (manual + automated).
Run cross‑line analysis (full survey).
Compute mean, std, histogram, and plot residuals vs depth.
Identify any systematic bias (>0.05 m) and correct (tide, datum).
If residuals vary with depth, re‑cast SVP and re‑process.
If residuals show heading‑dependency, re‑run patch test.
Apply 3‑sigma or MAD filter to identify outlier lines.
Re‑clean suspect lines (swath editor) and re‑run cross‑line.
Repeat until mean and std are within IHO limits.
Document final cross‑line statistics in report with colour map.

Click items to track progress (saved in browser).

8. Software & Tools

.htmlQPS Qimera那样Cross‑line analysis, statistical filter, CUBE outlier detection那样qps.nl/qimera.htmlCARIS HIPS那样Cross‑line verification, CUBE, residual maps那样teledynecaris.com.htmlHypack Processing那样Cross‑line QC module, line‑by‑line residuals那样hypack.com.htmlPython (numpy, scipy, matplotlib)那样Custom residual analysis, automated cleaning scripts那样Open source

Software	Cross‑line & statistical cleaning features	Link

9. Frequently Asked Questions

What is the acceptable percentage of cross‑line points that exceed the 95% confidence band?

IHO does not specify a percentage, but typically <5% of intersections should exceed ±2×TPU. If more, investigate.

How do I know if a residual is due to noise or a real seabed feature?

If a residual cluster corresponds to a known feature (e.g., wreck, rock), it is real. Otherwise, it is likely noise. Use side scan or backscatter to confirm.

Can I automatically reject lines with high cross‑line residuals?

Yes, in some software (e.g., Qimera), you can flag lines and automatically discard or re‑process them. But always review manually.

Why does my cross‑line std improve after SVP correction but still not meet IHO?

Check tide correction, patch test (especially roll), and GNSS vertical uncertainty. Also, check for residual heave errors.

What is the difference between cross‑line and overlap analysis?

Cross‑line compares perpendicular lines; overlap analysis compares adjacent parallel lines for coverage QC. Both are needed.

10. Action Items & Next Steps

📌 Use the cross‑line simulator: set std=0.18, outlier fraction 0.1 – run cleaning and note the improvement.
📌 In your processing software, export cross‑line differences as a CSV and create a residual map in QGIS.
📌 Identify the line with the highest residual and re‑inspect its swath editing.
📌 Proceed to Day 76: Final Product Generation – GIS & Charting.

📊 Day 75: Cross-Line Analysis and Statistical Cleaning