Camera Stream Gotchas

Post Reply
varghesesa
Posts: 61
Joined: Thu Jul 11, 2019 9:52 pm

Camera Stream Gotchas

Post by varghesesa » Thu Feb 04, 2021 3:18 pm

Introduction
The Camera Stream Gotchas article brings everything together. This article is about issues with the quality of your streams.

This article is not about connecting cameras. If you are having issues with connectivity like No Signal errors you should review the No Signal article in Cameras Troubleshooting. If you want to know how to connect cameras correctly review the Connect cameras correctly section in the Checklist.

This article explains the concept of BI video pipelines and explains how to easily troubleshoot issues yourself. For example, if playback is not working on your Android phone /tablet, you will know from this article that the video pipeline is

Recording -> Decode -> Encode -> Playback on device.

By knowing the pipeline, you can now tweak settings associated with each node (Record, Decode, Encode, Device) in order to get the issue resolved.

If you prefer to watch the webinar associated with this article, checkout the Camera Connections and Streams webinar.

Reference: If you want to learn more about how streaming and encoding works, a great reference article was created by IPVM titled "H.264 vs MJPEG - Quality and Bandwidth Tested". If you are not familiar with IPVM, they are a great resource for research and information regarding surveillance and security cameras.

Issue
We receive tickets associated with the camera streams being choppy, lagging in time, stuttering, loses signal or no signal error followed up with, however, the camera is up and working fine in the camera vendor's app.

The issue here is the vendor app is pulling streams from a proprietary port and protocol. The app is also an active solution, i.e. a user opened the app and wants to view a camera so the camera wakes up and delivers a quality, bandwidth optimized stream to the user.

Blue Iris (BI), like all VMS systems, is passively viewing all the time. Because we are a third party solution, we have access to different ports and settings on the available ports are crucial for BI to deliver a quality user experience.

Video Pipeline: Camera settings x BI Settings

In order for BI and your cameras to work well together, you have to take into account Camera and BI settings. The Video Streaming Pipelines webinar complements this section of the article. Once you understand the video pipelines, it becomes easy to understand the settings that can affect a particular pipeline and thus figure out why your video streams are not working.



Video Pipeline

All the Gotchas and fixes to gotchas further below are based on the combination of Camera settings x BI settings. In order to really understand the fix, understanding the video pipelines that exist in BI is pertinent. Once you understand the pipelines, it becomes much easier to know what effect the Camera settings x BI settings have on the pipeline. If you are curious to go deeper, reference the Blue Iris Streaming Overview article.


Image

*** Coming soon: Direct to wire ***
Bypass decoding / encoding completely. Send the camera encoding straight to remote endpoints. The remote device (mobile app, UI3, web site, YouTube etc) is responsible for decoding the stream successfully.



Gotchas: Video Artifacts

As you know, the Gotchas section documents learnings from past tickets. All the below Gotchas provide quick visuals of the issue. This is to allow users to quickly identify their issue and resolution. For those with more curiosity, details on the fix in the context of BI video pipelines is provided. This way users can attain a deeper understanding of BI video pipelines in order to self-diagnose future issues and resolve themselves instead of opening a ticket.


Console

Live view

Gotcha 1: Video distortion

Reported Issues:
  • Green bars as seen below.
    Encoding-HighToBaseline.png
    Encoding-HighToBaseline.png (52.99 KiB) Viewed 2644 times
  • Washed out image
    console-live-view-artifact.png
    console-live-view-artifact.png (95.3 KiB) Viewed 2644 times
The fix:

Video path: Camera video/feed -> Decode -> Console view
Camera feed: Simplify encoding. In this case, we changed the camera's encoding from H.264 High to H.264 Baseline. Alter Camera encoding settings until streams work.

Alternative possibilities: If camera settings did not resolve the issue, the next step would be to adjust the Decode settings.
Decode: Sometimes your hardware just cannot handle the camera's encoding. In this case the only option is to turn hwva off and decode the cameras via BI software.
Set Hardware decode = No in Camera settings -> Video tab.

Best practice is to always check Status -> Log for any errors.

Gotcha 2: Ghost images

Reported Issue:
Ghost images. See below.
ghost.png
ghost.png (174.73 KiB) Viewed 3097 times

Very common problem.

Ghost images occur when BI cannot decode the intermediate frames (p-frames / b-frames) coming from the camera. The image on the left, i.e. the actual scene was in the same mp4 video (video created on phone recording the console, not an mp4 export. to be discussed below) and would flash every 1s which was the key frame. In the ghost image (intermediate frame) you can kind of make out the mailbox behind the man walking. FYI, I enlarged the ghost image portion in the second image so you could see the correlation to the original scene. The image is not to scale.

The fix:

Video path: Camera video / feed -> Decode -> Console view
  • Decode: Turned off hwva.
    In this case, the encoding/decoding incompatibility was between the camera and Intel QuickSync. For some reason, the camera stats were saying the key frame ratio was 30 (mjpeg stream). It was almost as if the H.265 encoding from the camera was interpreted by the hwva as an mjpeg stream and thus rendering each intermediate frame as if it were a key frame. When hwva was turned off, the key frame ratio was 1 just as the camera settings show (GOP 1 I-frame / 1s).
As mentioned earlier, the user used his mobile device to create the video, not exporting to a file in BI. The user stated when exporting to a file (MP4), the video was fine. Big clue.

The video path for exporting to video is:
Read the file (recording) -> Decode -> Encode -> Save to MP4.

I also know the default export setting is to re-encode using BI software (i.e. skip hwva). Thus hwva is the only difference between the two video paths, so hwva must be causing the issue.

Playback

The biggest difference between Playback vs Live view is the video source is coming from the recording. Make sure Live view is working first.
Garbage in = Garbage out.

Video pipeline for console playback:
Recording (BVR) -> Decode -> Playback window.

This means the recording format (Re-encode / D2D) comes into play. This also means the hard drive is also in play since the feed comes from a file recording.


Gotcha 1: Playback is a black screen.

playback black screen.png
playback black screen.png (16.47 KiB) Viewed 2062 times

Reported issue: Usual ticket states Live view is fine but playback has a black screen.

This issue may also pop up if you happen to click on a file that is currently open for recording (red border around it). If you are recording Direct to Disk (D2D) and the key frame interval is sub-par, e.g. <1, then it is possible the file is open for recording, but no key frames have actually been recorded yet so BI cannot play back.

Hardware decode
It's possible that your HA has problems decoding a clip. HA works well with steady streams (live view) because the buffer can fill up and the hardware can process frames at a steady rate. With playback, users can go back and forth in the video using the playback controls. This use case is not conducive in general for HA so turning off HA during playback is an easy check. Unselect Also BVR. Camera settings -> Video tab.


Camera setup
Check your camera stats, specifically the key frame interval. Camera settings -> General tab. If the key frame interval is <1, go to your settings on the camera and raise the key frame interval. Camera setup article has details.

Maybe simplify the camera encoding settings as well. D2D should work with playback if live view works. Changing the camera encoding affects the contents being saved to the file when choosing D2D. This is how you indirectly affect the Recording node by changing camera settings.

If you cannot increase the key frame interval, provide an adequate pre-trigger buffer. See BI Server settings article for details.


Record
D2D recordings is very popular. It's possible Also BVR is unselected and you are receiving a black screen during playback. The Live view works fine. This could mean you are using HA for the live view and your hardware can handle the camera encoding. However, the software decoding for playback (Also BVR unselected) can not handle the camera encoding (saving D2D).

The Fix:
Change the Recording from D2D to Re-encode.
Camera settings -> Record tab -> Video file format and compression.
Create the BVR file using BI software for encoding instead of the camera encoding.
This will lead to a bump in CPU. Monitor usage.


Mobile app
First confirm live view and playback work from the console. There is no point trying to troubleshoot video on the mobile app unless the console is working.
Garbage in = Garbage out.

Live view

Gotcha 1: Video distortion

mobile streaming.png
mobile streaming.png (138.06 KiB) Viewed 2225 times

Issue: Encoding from BI is too complicated for your mobile device.
Video pipeline: Camera video/feed -> Decode -> Encode -> Mobile app
As soon as you introduce a remote endpoint, the Encode node in the video pipeline is introduced.

Fix: Adjust encoder settings. Global settings -> Web server tab -> Advanced -> Configure.
  • Set quality to 50%.
  • Hardware acceleration = No. Hardware encoding almost never works with mobile media players. Best to encode in BI software.
  • In the Advanced section, change Profile to main.
encoder options.png
encoder options.png (44.06 KiB) Viewed 2663 times

Gotcha 2: Camera Stream Issues

If you believe an update is causing the issue, go back to a previous version, confirm issue goes away and let us know. That is valuable information and may help resolve the issue. However, we may still need to troubleshoot in your environment to understand why the current version does not work.

Isolate the issue

The question here is whether the issue is with the network or the mobile app. To rule out the network, open a web browser on the mobile device and try to connect and login to the web server. Is the web interface working well? If not, the issue resides with your BI server or the network.

Server
An easy way to test whether your server is the bottleneck is to turn the Shield to red. You have shutdown many core functions on the server including trigger, record, alert. Does streaming to the endpoint now work better? Then you know your CPU is working too hard. Otherwise you will need to troubleshoot your network.

Mobile app
If UI3 works fine on the mobile device, then the issue must reside with the video pipeline.
Camera feed -> Decode -> Encode -> Media player on remote device

A common problem is the encoder settings. The above settings generally work across media players.


Playback

The biggest difference between remote endpoint playback (mobile app, UI3) vs console playback is the Encoding node.
Video Pipeline:
Recorded file -> Decode -> Encode -> Mobile device

Before troubleshooting remote endpoint playback, confirm console playback is working. See above.
Garbage in = Garbage out.

Gotcha 1: Playback is a black screen when playing on a remote endpoint (web interface, mobile app)

Reported issues: Playback at a remote endpoint is not working. Other ticket descriptions below.
  • Playback is a gray screen. Playback is a black screen.
  • Randomly restarts from the beginning. Playback hangs.
ios gotcha.png
ios gotcha.png (33.51 KiB) Viewed 2227 times


First confirm console playback is working. If so, encode settings is the only node left that could be causing issues.

Fix: Simplified encoding settings. Global settings -> Web server -> Advanced -> Configure (Stream 0)
The Live view section above has a screenshot of Encoder settings that usually works. Most important settings:
  • Hardware acceleration = No. Critical fix.
  • Maybe limit bit rate will have an effect.
  • Maybe Resize output frame width x height will have an effect.


Gotcha 2: Playback on Mobile device is not smooth

Similar to above, the Encoding settings is often the culprit.
The issue could also be with the video player on the device. For example, we have seen tickets where playback was fine on a Samsung galaxy and broken on a Pixel 6. A good test beyond simplifying encoding settings is test whether you can connect to the server from the mobile web browser. If the UI3 interface works then the culprit could be the video player. Browsers use their own video players.


File Export

Export to file has the same complexity as Playback with ONE more layer of complexity, the encoding settings similar to remote access.
Video Pipeline: File recording -> Decode -> Encode -> File (usually mp4)

Gotcha 1: Export to file
Pulsating jaggies happening at consistent intervals, e.g. every 1s.

pulsating jaggies.png
pulsating jaggies.png (200.56 KiB) Viewed 3102 times
Reported issue:
The user was attaching a 10s MP4 file with their email alerts.

Symptom: The video starts out fine but consistently every 1s or 2s, i.e. key frame interval, the image gets very pixelated. The key symptom is the fact this happens in a consistent interval like every 1s or 2s implying BI or the stream is having issues processing the key frame.

The Fix:

Video path: Recorded file -> Decode -> Encode -> Save to file (MP4)
  • Encode: Turn off hwva.
    We got lucky. As you can see, the video path has many hops that could cause issues.
    Starting from the end and working backwards is always a good strategy.

Streaming Artifacts

Examples of Stream quality issues
There are so many types of issues dealing with good healthy streams that we decided to create a separate section. This section focuses on live view at the console. These gotchas are more about the video quality being smooth, i.e. not choppy, jittery etc. The resolution often has to do with camera encoding settings.

Console

Live view

Gotcha 1: Choppy streams
Choppy streams, i.e. the live view or the playback is not smooth or the seconds on the time overlay are not sequential. These issues are almost entirely due to camera encoding settings. If the cameras are speaking Spanish and BI is speaking English, neither party can understand each other resulting in a poor experience.

The Fix:

Video path: Camera video -> Decode -> Console view
  • Camera video: Check out the camera stats, Status -> Cameras tab. The FPS should be at least 15 fps.
  • Camera video: Look at the key frame ratio in Camera -> Stats. Anything less than 0.5 may provide a bad experience. Set Key frame interval = fps, e.g. 15.
  • Revisit Camera setup article.
Gotcha 2: Camera streams lagging or sluggish or high latency

Fix:
Video path: Camera video -> Decode -> Console view

Almost always, camera streams are sluggish because BI is not receiving enough key frames (i-frames). Need to alter encoding settings on the camera.
As stated above, key frame ratios less than 0.5 lead to sluggish cameras.
Revisit Camera setup article.


CPU maxed out

The other reason for lagging camera feeds in live view is the cameras are high resolution and the BI server cannot keep up. If your CPU is running above 80%, it is working too hard. In my opinion, above 50% is concerning. My CPU with (4) 2 MP cameras (I know, not much) hovers around 8%.

Below are the general steps to tune your system if you want to confirm everything is setup optimally.
Gotcha 3: Stuttering at common intervals, e.g. every 1 or 2s.

The fix:

Video path: Camera video/feed -> Decode -> Console view

Camera feed: Change stream setting from Constant Bit Rate (CBR) to Variable Bit Rate (VBR)
Suppose a key frame is sent from the camera every 1s. If the CBR setting is too low, then the camera needs to perform image optimizations to get the frame across the network. Those optimizations lead to the glitches. So simply changing the camera setting to VBR instead of CBR should resolve the issue. In general, setting your cameras with VBR is a better user experience. CBR should only be considered if you have network bandwidth issues.


Tips and Tricks

Pro tip 1: Tablet or TV on the LAN.

Many users are using TVs and Tablets to display their cameras. Since these devices are usually on the LAN, they want to take advantage of their high resolution screens with high resolution video streams, since they know their network can handle the load.

Video Pipeline: Camera video / feed -> Decode -> Encode -> Endpoint (large TV or high resolution tablet)
Fix: Details below, but basically adjust the Encode settings to provide a high quality stream to the endpoint.

ui3 tv.png
ui3 tv.png (184.36 KiB) Viewed 2231 times

To get high resolution, smooth video, try the following settings for one of your web server video streams.
  • Use variable bit rate instead of constant bit rate. Unselect Rate control. Crank up Quality value. Gotcha: If network bandwidth too high, adjust.
  • Profile = high Uses more bells and whistles such as b-frames to optimize bandwidth while delivering quality. Gotcha: If endpoint cannot decode, adjust.
  • Preset = veryfast. Set to lowest to maximize quality. Gotcha: If CPU utilization is high adjust.
  • Uncheck Resize output frame. Send the full camera resolution to the device. Gotcha: If network bandwidth spikes, adjust. If the player at the endpoint starts to choke, adjust
  • b-frames are great for increasing quality and optimizing bandwidth. Gotcha: However, if you have a PTZ camera, b-frames can cause delays displaying the new view after a camera position changes.
LAN settings 320.png
LAN settings 320.png (73.01 KiB) Viewed 2231 times

Don't forget to choose the appropriate stream for the web interface or the mobile app. Below is a reminder on how to do so on a tablet.
Home tab -> Gear icon -> LAN. Profile 0 is the first stream, profile 1 is the second stream and profile 2 is the third stream.

ios settings 320.png
ios settings 320.png (45.01 KiB) Viewed 2231 times

One final comment. If you have a monitor or TV that does NOT have internet connectivity a popular solution is to purchase an Amazon Firestick and run UI3 from there to the monitor.


Pro tip 2: Leverage UI3 to help set optimal encoding settings on your mobile device

The encoder dialog box is a bit daunting. What are good values for all those settings?
If like most, you are not well versed on encoder settings, use the UI3 interface for help!

The issue here is if you bring up a browser on your phone and go to the BI web interface, playback and live view work fine. When doing the same through the app, the streaming is choppy, jittery etc.
  • Test 1: Need to make sure live stream and playback are first smooth on the BI Server. Garbage in = Garbage out.
  • Test 2: If you open a browser on the phone/tablet and go to the BI web interface (UI3), are the live view and playback a good experience. If so, then you know the problem is not with the network.
If you are still reading, then the problem probably means the player used by your device cannot handle the incoming stream well. The short answer is to go to Global settings -> Web server -> Advanced -> Encoder profiles (Choose Streaming 0 or 1) -> Configure dialog. Use the below settings to start and see if they work.


encoder options.png
encoder options.png (44.06 KiB) Viewed 2663 times

Below are the changes from the default settings:
  • First, I turned on Constant Bit Rate (CBR). This settings caps the amount of data going to the player at 1024 kbps.
  • Next, I reduced the resolution to 856 x 480. You are streaming to a mobile device, so reducing the resolution will probably have no affect on your viewing experience. I played with "Retain aspect ratio within frame". On iOS devices, the setting seemed fine. But I noticed instability and crashes on some Android devices. It does not really change/help the user experience, so my conclusion was to just leave it off unless required for another reason.
  • Finally, I set Hardware encode = No. The hardware encoding often will not play on mobile players. Best to encode using BI software.
Reference: If you want a deeper understanding of the settings in the encoder dialog or how encoding generally works, a great reference article was created by IPVM titled "H.264 vs MJPEG - Quality and Bandwidth Tested". If you are not familiar with IPVM, they are a great resource for research and information regarding surveillance and security cameras.

So how did I choose the values for the bit rate and resolution?

I cheated and used the expertise of the BI development team! If you click on the gear icon in UI3 (lower right), you can select any predefined resolution and hit the pencil button up top. You then see the encoding settings for the selected resolution. I just copied those values into the encoder dialog on the console.

ui3-streaming-settings-trick_smaller.png
ui3-streaming-settings-trick_smaller.png (100.61 KiB) Viewed 1755 times

Once I made the stream adjustments, I went to the mobile app settings and chose the appropriate settings for my LAN and WAN connections.

video streaming gotcha smaller.png
video streaming gotcha smaller.png (22.98 KiB) Viewed 1755 times

Next steps / Submitting a Ticket

If still an issue put the camera on the WAN for remote testing.
Post Reply