Day 71: Quality Control & Data Validation – Masterpiece Edition | River Warrior

✅ DAY 71: QUALITY CONTROL & DATA VALIDATION

⏱️ Estimated Reading Time: 17 Minutes | 🎓 Level: Professional Hydrographer / QC Specialist

Ensuring Every Sounding Meets IHO Standards – QC Workflows, Cross‑Line Analysis, and TPU Validation

Instructor: Engr. Rokib Hossain | River Warrior Academy

🏠 Course Homepage

📖 Table of Contents (Serialised)

Why Quality Control Is the Final Gatekeeper
QC Workflow: From Acquisition to Final Product
Cross‑Line Analysis Revisited (IHO Acceptance)
Total Propagated Uncertainty (TPU) Validation
Interactive Cross‑Line & TPU Simulator
Outlier Detection & Statistical Filtering
Metadata Validation (Datums, Projections, Tide, SVP)
Case Study: Bay of Bengal QC Failure & Recovery
QC & Data Validation Checklist
Resources & Software
Frequently Asked Questions
Action Items & Next Steps

1. Why Quality Control Is the Final Gatekeeper

No hydrographic survey is complete without rigorous QC. Quality control verifies that the data meets project specifications (IHO S‑44, client requirements) and is free from systematic errors. QC happens at multiple stages:

✅ Real‑time QC during acquisition (Day 19).
✅ Post‑processing cleaning and cross‑line analysis (Day 47).
✅ Final validation before delivery.

Skipping QC can lead to rejected deliverables, legal disputes, and safety hazards.

🧠 Golden Rule: QC is not a one‑time event; it is a continuous process. Every processed line should be checked before moving to the next.

🌊 River Warrior Pro-Tip: QC Saves Millions

During a Bay of Bengal pipeline route survey, a post‑processing QC check revealed a 0.3 m vertical shift in one swath due to an uncorrected tide offset. Reprocessing before delivery saved the client from a pipeline misalignment that would have cost $2M.

2. QC Workflow: From Acquisition to Final Product

1️⃣ Real‑time: GNSS fix, motion, coverage, SVP age

2️⃣ Raw data integrity: checksums, file sizes

3️⃣ Navigation & tide correction verification

4️⃣ Cleaning & outlier rejection (swath editor)

5️⃣ Cross‑line analysis (mean difference, std dev)

6️⃣ TPU computation and validation

7️⃣ Surface QC: hillshade, contours, anomalies

8️⃣ Metadata & report generation

3. Cross‑Line Analysis Revisited (IHO Acceptance)

Cross‑line analysis (Day 47) is the most important QC metric. It compares depths from main survey lines with perpendicular cross‑lines at intersection points. IHO S‑44 requirements for Order 1a:

Mean difference (bias) ≤ 0.1 m.
Standard deviation ≤ 0.15 m.
95% of differences within ±0.3 m.

If cross‑line statistics exceed limits, investigate SVP, tide, or patch test errors.

📊 For deep water (>100 m), scaling applies: allowable std dev = 0.5% of depth. For 500 m depth, std dev ≤ 2.5 m.

4. Total Propagated Uncertainty (TPU) Validation

TPU quantifies the theoretical depth uncertainty from all error sources (GNSS, motion, SVP, sonar). QC must verify that measured cross‑line differences align with predicted TPU. Steps:

Compute TPU per sounding using IHO formula (a² + (b×depth)²).
Compute average TPU over the survey area.
Compare cross‑line standard deviation to predicted TPU (should be ≤ TPU).
If cross‑line std > TPU, there are unmodelled errors.

🔬 Many clients require a TPU compliance certificate alongside the final chart.

📊 QC Simulator: Cross‑Line & TPU Check

Enter cross‑line difference statistics and TPU to evaluate IHO compliance:

Mean difference (m): Standard deviation (m): Average TPU (m):

Cross‑line: PASS (std=0.12 ≤0.15) | TPU: PASS (std ≤ TPU)

5. Outlier Detection & Statistical Filtering

Outliers that survive cleaning must be caught by statistical QC. Methods:

3‑sigma rule: Reject points with depth >3 standard deviations from local median.
Median absolute deviation (MAD): More robust than std.
CUBE (Combined Uncertainty): IHO‑recommended algorithm that flags outliers based on uncertainty.

After applying, re‑run cross‑line analysis to ensure no bias introduced.

🔧 In Qimera, use the “Statistical Filter” with a 3‑sigma threshold. Always review flagged points manually.

6. Metadata Validation (Datums, Projections, Tide, SVP)

Even perfect depths are useless if the metadata is wrong. QC must verify:

Horizontal projection matches project requirements (e.g., UTM zone).
Vertical datum: Chart Datum (LAT) or MSL; geoid model used.
Tide correction: gauge location, datum offset, time synchronisation.
SVP files: cast times, max depth, applied correctly.
Patch test values: roll, pitch, yaw, latency recorded.

📌 Create a metadata checklist that must be signed off before data processing begins.

7. Case Study: Bay of Bengal QC Failure & Recovery (2026)

Scenario: After processing a 200 km² MBES survey, the cross‑line standard deviation was 0.28 m (exceeding Order 1a limit of 0.15 m).

QC investigation: Checked SVP casts – one cast was 8 hours old, located 30 km from survey area. Re‑applied correct SVP (new cast).
Result: Cross‑line std dropped to 0.11 m, TPU validation passed.
Lesson: A single bad SVP can ruin QC. Always log SVP metadata and verify spatial representativeness.

📈 The QC report included a “before/after” cross‑line difference map, demonstrating the improvement – essential for client confidence.

8. QC & Data Validation Checklist

Real‑time QC logs reviewed (GNSS fix, motion, SVP age).
All raw data checksums verified.
Cross‑line analysis performed – mean and std within IHO limits.
TPU computed and compared with cross‑line std (should be ≤ TPU).
Outlier filtering applied (3‑sigma or CUBE) and manual review done.
Metadata validated: projection, datum, tide, SVP, patch test.
Surface inspected for artefacts (hillshade, contours).
Volume vs design checked (for dredge projects).
Final QC report generated with statistics and summary.
Client sign‑off obtained.

Click items to track progress (saved in browser).

9. Resources & Software

.htmlCARIS HIPS那样CUBE, TPU validation, cross‑line那样teledynecaris.com.htmlHypack Processing那样Cross‑line QC module, report generator那样hypack.com

Tool / Resource	QC Features	Link
QPS Qimera那样Cross‑line analysis, TPU, statistical filtering那样qps.nl/qimera
NOAA Field QC Guidelines那样Standard QC procedures那样NOAA PDF

10. Frequently Asked Questions

What is the difference between quality control (QC) and quality assurance (QA)?

QC is the process of checking data quality (e.g., cross‑line analysis). QA is the system of procedures that ensures quality is built into the survey (e.g., equipment calibration, training). Both are necessary.

How often should cross‑line analysis be performed?

After cleaning of each batch of lines, and again after full processing. For large surveys, run cross‑lines after every 10‑20 main lines.

What TPU calculator should I use?

IHO provides a spreadsheet; CARIS and Qimera have built‑in TPU propagation. Use uncertainty budgets for each sensor (GNSS, IMU, SVP).

My cross‑line std is 0.18 m for Order 1a (shallow). What should I do?

Check SVP (re‑cast if >4 hours), verify tide correction, re‑run patch test. If still high, investigate motion sensor or GNSS.

Can QC be automated?

Partially: cross‑line statistics, TPU comparison, and outlier filtering can be scripted. However, human review of surface anomalies is still necessary.

11. Action Items & Next Steps

📌 Use the QC simulator: enter mean=0.05, std=0.18, TPU=0.14 – see what fails.
📌 Review a previous survey’s cross‑line report; if it failed IHO, identify the cause.
📌 Create a QC checklist template for your next project.
📌 Proceed to Day 72: Multibeam Patch Test Calibration.

🛡️ Day 71: Introduction to Module 12 – Quality Control & Data Validation