PIE-PS Photometric Stereo from Physical Irradiance Event Streams
Dense surface-normal reconstruction from raw event streams captured under moving illumination.
Overview
Recovering surface orientation from asynchronous visual measurements
Event cameras provide microsecond-level measurements and high dynamic range, but their sparse outputs depend on an unknown contrast threshold. PIE-PS learns to recover dense surface normals directly from raw events and known moving-light directions.
The framework builds physical irradiance observations from adjacent events, shares information across space and time, and learns to reduce the influence of unreliable measurements. Experiments on synthetic benchmarks and real captures consistently improve over prior event-based photometric stereo methods and a direct physics-only solver.
Video Results
Moving illumination, asynchronous events, and the reconstructed normal map are shown side by side as observations accumulate.
Qualitative Comparison
Normal predictions are paired with angular-error maps using the same 0°–30° scale. Darker error maps indicate lower error.
Quantitative Results
Mean angular error (MAE) in degrees. Lower is better.
DiLiGenT-Sim
| Method | Ball | Buddha | Cat | Cow | Goblet | Harvest | Pot1 | Pot2 | Reading | Average |
|---|---|---|---|---|---|---|---|---|---|---|
| EventUPS | 8.86 | 12.21 | 10.78 | 18.58 | 12.97 | 23.32 | 8.70 | 15.13 | 16.71 | 14.14 |
| EventPS-OP | 10.99 | 18.73 | 12.74 | 26.51 | 18.43 | 36.06 | 13.78 | 15.75 | 24.61 | 19.73 |
| EventPS-FCN | 7.49 | 18.13 | 11.42 | 20.61 | 18.07 | 26.05 | 12.83 | 16.59 | 15.16 | 16.26 |
| EventPS-CNN | 10.44 | 16.79 | 11.88 | 20.60 | 16.44 | 25.26 | 12.93 | 15.54 | 18.19 | 16.45 |
| Direct Solver | 15.36 | 15.09 | 13.75 | 14.55 | 15.62 | 20.07 | 13.59 | 14.67 | 17.72 | 15.60 |
| PIE-PS | 3.93 | 8.67 | 5.15 | 12.90 | 7.81 | 12.91 | 6.33 | 8.34 | 12.25 | 8.70 |
PIE-Sim
| Method | Blobby Circle |
Blobby Hypo. |
Blobby DiLiGenT |
Sculpture Circle |
Sculpture Hypo. |
Sculpture DiLiGenT |
Average |
|---|---|---|---|---|---|---|---|
| EventPS-FCN | 28.83 | 16.83 | 21.73 | 31.63 | 26.43 | 29.59 | 25.84 |
| EventPS-CNN | 43.50 | 42.11 | 26.08 | 48.31 | 46.85 | 28.08 | 39.15 |
| Direct Solver | 27.08 | 17.24 | 13.02 | 26.60 | 24.68 | 22.67 | 21.88 |
| PIE-PS | 10.68 | 6.41 | 4.81 | 17.17 | 12.09 | 12.77 | 10.66 |
Real-world Scenes
| Method | Real-1 | Real-2 | Real-3 | Bunny | Rectangle | Flower | Average |
|---|---|---|---|---|---|---|---|
| EventPS-FCN | 15.79 | 19.65 | 9.42 | 26.14 | 18.70 | 35.76 | 20.91 |
| EventPS-CNN | 17.65 | 20.16 | 14.85 | 48.78 | 48.11 | 43.14 | 32.12 |
| Direct Solver | 21.08 | 21.10 | 20.50 | 23.46 | 32.22 | 25.02 | 23.90 |
| PIE-PS | 6.51 | 6.09 | 5.92 | 12.69 | 11.52 | 19.57 | 10.38 |
Resources
Code, data, and the pretrained checkpoint are available below.
Paper (arXiv) Code Data Pretrained checkpoint
Citation
BibTeX will be added with the final publication record.