Introduce
- All in one (Client and Server integrated one container) app for edge streaming with NPU
Specific features
- Used unofficial Gstreamer API for Node Js
- Client and Server is integrated, Edge computing effective
Results
Struggles
Issues Arising During Pipeline and Frame Transmission
- Handling Image Frame:
Various formats have been tested, such as RGB raw and
Uint8Array, but mismatched image dimensions can result in corrupted outputs like the ones below. - Python Solution: These issues can be efficiently handled using OpenCV.
- JavaScript Challenges: Due to a lack of suitable libraries, the image format used in transmission had to be altered to ensure compatibility.
- Transformation Process:
base64 buffer → Uint8Array (R, G, B) → JPEG → **Blob (video/webm; codecs=vp8)**
- Common Problems:
- Mismatched resolutions.
- Using incorrect channel formats, such as RGB with 4 channels.