Day 26: Processing Introduction – Masterpiece Edition | River Warrior

⚙️ DAY 26: PROCESSING INTRODUCTION

Masterpiece Edition – From Raw Data to Final Bathymetric Products

Instructor: Engr. Rokib Hossain | River Warrior Academy

🏠 Course Homepage

📖 Table of Contents (Serialised)

Why Processing Matters
Hydrographic Processing Workflow
Data Conversion & Loading
Data Cleaning (Swath Editing)
Surface Generation & Gridding
Total Propagated Uncertainty (TPU)
Final Product Creation (Charts, Volumes)
Processing Checklist
External Resources & Software
Frequently Asked Questions
Action Items & Next Steps
Post‑Verification Checklist

1. Why Processing Matters

Raw hydrographic data contains noise, outliers, and systematic errors. Processing transforms raw measurements into a clean, validated bathymetric surface that meets IHO S-44 standards. Without proper processing, even the most expensive sensors produce unusable charts.

Processing is a multi‑step workflow that typically uses dedicated software (e.g., Qimera, CARIS HIPS, Hypack Processing, MB-System). This day provides the conceptual foundation; later days (27–45) dive deeper into each step.

🧠 Golden Rule: Process data in the same coordinate system and datum used during acquisition. Never skip the TPU (Total Propagated Uncertainty) calculation – it is required for IHO compliance.

2. Hydrographic Processing Workflow

High‑level processing workflow: raw → loaded → cleaned → gridded → final product.

1️⃣ Import raw data (ALL, S7K, RAW)

2️⃣ Apply corrections (tide, SVP, attitude)

3️⃣ Reject outliers (swath editor)

4️⃣ Compute TPU & cross‑line stats

5️⃣ Generate surface (CUBE, weighted average)

3. Data Conversion & Loading

Most processing software includes a “Loader” or “Import” module. Raw files from different manufacturers (e.g., Kongsberg .all, Teledyne .s7k, R2Sonic .son) are converted to a common internal format. During loading, the software applies:

Geodetic parameters: Projection, datum, geoid model.
Sensor offsets: Lever arms (GNSS to transducer).
Calibration values: Patch test results (roll, pitch, yaw, latency).
Corrections: Tide, SVP ray‑bending, heave.

Always verify that the loaded data aligns with the expected survey boundaries and that no sensor is missing (e.g., IMU data not recorded).

💡 Tip: Use a small subset (e.g., one line) to test the loading parameters before processing the entire survey.

4. Data Cleaning (Swath Editing)

Cleaning is the process of removing erroneous soundings (outliers) caused by noise, bubbles, fish, or false bottom detections. Most software provides a “swath editor” with these tools:

Manual point rejection: Click on individual outliers.
Auto‑filter by slope: Reject points with unrealistic depth change.
Statistical filter: Remove points beyond 3‑sigma from median.
Beam‑pattern filter: Reject outer beams if SNR is low.

Red spike = outlier to be rejected. After cleaning, the profile becomes smooth.

5. Surface Generation & Gridding

After cleaning, the scattered soundings are gridded into a regular bathymetric surface (grid). Common gridding algorithms:

CUBE (Combined Uncertainty and Bathymetry Estimator): Statistically robust, IHO recommended.
Weighted average: Simple, but less resistant to outliers.
Nearest neighbour: Preserves original soundings but may create artefacts.

Grid resolution (cell size) is chosen based on survey order (e.g., 0.5 m for Order 1a in shallow water). Finer grids show more detail but require more cleaning.

📊 Pro tip: Create multiple surfaces: a high‑resolution (e.g., 0.5 m) for feature detection and a lower resolution (e.g., 2 m) for volume calculations.

6. Total Propagated Uncertainty (TPU)

TPU quantifies the depth uncertainty of each sounding based on contributions from:

GNSS position error (horizontal & vertical).
Motion sensor (heave, roll, pitch) uncertainty.
Sound velocity profile error.
Echosounder measurement noise.

IHO S-44 requires TPU to be computed and visualised. In processing software, you define standard deviations for each sensor (e.g., 0.02 m for GNSS vertical). The software then computes a confidence layer for the final surface.

🔬 Field application: TPU values >0.5 m may indicate a problem with your SVP or motion sensor. Re‑cast SVP or check IMU calibration.

7. Final Product Creation (Charts, Volumes)

From the cleaned grid, you generate deliverables:

Bathymetric contour map (DWG/PDF).
Volume calculation (cut & fill).
ENC (Electronic Navigational Chart) – S-57/S-101.
Georeferenced backscatter mosaic.
Cross‑line difference report.

Most processing software includes export modules to CAD, GIS, or chart formats. Ensure the final product includes a metadata report (date, vessel, sensors, processing steps, TPU).

8. Processing Checklist

✅ Raw data integrity confirmed (file sizes, checksums).
✅ Corrections applied: tide, SVP, heave, alignment (patch test).
✅ All lines loaded without sensor gaps.
✅ Swath cleaning completed (outliers removed).
✅ Cross‑line analysis performed (difference < IHO tolerance).
✅ TPU computed and visualised.
✅ Grid generated with appropriate cell size.
✅ Final products exported and archived with metadata.

9. External Resources & Software

Software	Purpose	Link
QPS Qimera	MBES processing, CUBE, TPU	qps.nl
CARIS HIPS/SIPS	IHO chart production, TPU	teledynecaris.com
Hypack Processing	Cleaning, surface, volume	hypack.com
MB-System (open source)	Command‑line processing	mbari.org
NOAA CUBE Tutorial	CUBE theory and practice	NOAA CUBE

10. Frequently Asked Questions

🔹 What is the difference between cleaning and filtering?

Cleaning typically refers to manual or semi‑automatic removal of outliers in swath editor. Filtering (e.g., median filter) is applied after gridding. Both are needed.

🔹 Why do I need TPU if my data looks clean?

TPU is a requirement for IHO compliance. It provides a statistical confidence layer. Even clean‑looking data may have hidden systematic errors.

🔹 How much cleaning is too much?

Never reject more than 5‑10% of soundings. If you need to reject more, your acquisition parameters (SVP, gain) are likely incorrect. Review Day 24.

🔹 Can I process MBES data without a dedicated software?

Yes, using open‑source MB-System, but the learning curve is steep. For professional work, Qimera or CARIS is recommended.

🔹 What is the typical grid cell size for a port survey?

For IHO Order 1a in depths <10 m, 0.5 m cells. For deeper water (30‑50 m), 1‑2 m cells.

11. Action Items & Next Steps

📌 Download a sample MBES dataset (e.g., from NOAA or QPS demo) and try loading it into a free viewer.
📌 Identify the processing software you will use (Qimera trial, Hypack, or MB-System).
📌 Write a one‑page processing workflow for a typical port survey.
📌 Proceed to Day 27: Data Conversion & Navigation Correction.

12. Post‑Verification Checklist

✅ Canonical URL & meta tags correct.
✅ Schema includes isAccessibleForFree: true.
✅ TOC uses numbered list with anchors.
✅ Internal links to Day 24, Day 27 work.
✅ Print CSS hides interactive buttons.
✅ FAQ accordion functional.

📊 Day 26: Introduction to Module 4 – Data Processing & Cleaning