Multispectral WebGL

One of those very technical talks from

Iván Sánchez Ortega

Brief bio

  • Nerd
  • CompSci dropout
  • Bought a GPS in 2006

Prelude

I don't like Zoom.

Not FLOSSy, security & data privacy concerns

Try Jitsi.org!

color colour

Spectrum visible colour bands

 

Whole spectrum

https://commons.wikimedia.org/wiki/Sinclair_ZX_Spectrum#/media/File:ZXSpectrum48k.jpg

Chapter I:

GIS vs videogame tech*

* = OpenGL/WebGL, but concepts also apply to DirectX & Vulkan

Both fields are multidisciplinar within CompSci

  • Sensors
  • Topology
  • Data storage
  • Concurrency
  • Economics
  • Heuristic AI

Both fields have acronyms that make no sense

EPSG

European Petroleum Survey Group
Datum-Projection code

VBO

Vertex Buffer Object
Attribute Write-Only Memory Allocation

Both fields have weird coordinates

  • equirectangular
  • traverse cylindrical
  • WGS84, ETRS89, CRS80
  • Lambert conical
  • Null Island
  • world coordinates
    e.g.(0,65535), north-east-up
  • clipspace coords
    (-1,+1), up-right-deep
  • screen (pixel) coords
    e.g. (0,1024) (0,720), down-right
  • normalized screen
    (0,1), down-right (OpenGL), up-right (DirectX)
  • texture coords
    (0,1) axis direction varies
https://songho.ca/opengl/gl_projectionmatrix.html

In both, pixels are more than RGB

  • Short-Wave InfraRed
  • Near-InfraRed
  • X-ray reflectivity
  • Panchromatic
  • Elevation
    (with tree canopy fun!)
  • Slope
  • Alpha
    (opacity)
  • Albedo
    (RGB minus own shadows)
  • Occlusion
    (shadows)
  • Bump map
    ("normal map"/"slope")
  • Specular map
    (reflectivity)
  • Gloss map
    (reflectivity sharpness)
  • Motion/blur
  • Slope

In both cases, the (RGB) colour in the screen is a function of the data for each pixel...

...so videogames are doing raster algebra, after all.
https://www.shadertoy.com/view/Mdl3WH

The big difference

GIS → Accuracy
«Acc worse than 0.5cm is garbage rubbish»
Games → Speed
«OMG this game doesn't run at 60FPS 4K on my 100Hz HDR 46' ultrawide screen»

Convergence & opportunity

GIS is faster because tiles
  • WMTS
  • Vector Tiles
  • Cloud-Optimized-GeoTIFFs
GPUs are more accurate because games bitcoin AI
convolution-driven neural networks to perform image recognition on cat photos
  • 8-bit RGBA textures → 24-bit floats → 32-bit float data cubes
  • Ubiquitous (WebGL > 98%, WebGL2 ~75%)

Sweet spot:

  • Get a raster tile...
  • ...with several 8- or 16- or 32-bit data samples per pixel...
  • ...put it into a texture...
  • ...and let the GPU turn those texture(s) into a RGBA colour value
Result: real-time (~30-60Hz) raster algebra
Controversial fact:

Videogames have been doing raster algebra better and a hundred times faster than any GIS software for years.

Challenges

GIS has a lot of baggage
  • Tool-specific languages for raster algebra
    (R, GDAL, sentinel-hub, ESRI)
  • Raster-as-a-service
    (more rasters served = more invoices)
  • Metadata
    (metadata)
GL tech is deeply entrenched in high-level frameworks
  • Models, scene trees, materials
    (but this just needs two triangles)
  • No (or subpar) tooling/debugging for low-level GL work
  • C-level-like magic
    (e.g. recompiling shaders resets all attribute, uniform and texture units to zero, except the 0th)
  • Obstuse documentation
    (300-pages-long spec in PDF? thanks, no thanks)
Chapter II:

The secret to hacking web maps

  • Analysis
  • Morphosis
  • Synthesis

  • Break it up
  • Change a part
  • Put it back together

  • Be aware of all the moving pieces
  • Change one specific moving piece
  • Ensure everything keeps moving together

https://sci-hub.tw/https://iopscience.iop.org/article/10.1088/1742-5468/2009/06/P06016/fulltext/
 
 

State of the art

Working proofs-of-concept:

  • Leaflet.TileLayer.GL
    • WebGL1, max 8 RGBA 4×8-bit textures
  • Leaflet.TileLayer.GL2
    • WebGL2, max 8 2048×2048×2048 float-32 data cubes
    • geotiff.js support
  • OpenLayers' GlTiles tile source
    • WebGL1, max 8 RGBA 4×8-bit textures
    • geotiff.js support for 8-bit geotiffs
    • Really soon™: Uint16 support via byte packing

Issue: In-CPU resampling

  • GeoTIFF tiles BBOXes are relative to GeoTIFF BBOX
  • GeoTIFF tiles don't fit (pseudo-)WMTS tiles
  • Resampling needed

Approach 1: Skip reusing tile schemes

  • Tile bbox queries for geotiff.js?
  • In-GPU resampling?
  • Redo tile visibility logic

Approach 2: WebGL in geotiff.js

  • Improve geotiff tile caching
  • Blind WebGL development
  • In-geotiffjs shaders (at least for resampling methods)
Chapter III:

The big scary demos

Fin.
¿?