Camera Stream Optimization

Details on the five major steps to configure and deploy Blue Iris.
1. Connecting IP cameras.
2. Configuring your home or office network for Remote Access via a web browser or the phone apps.
3. Setting up Storage and Recordings.
4. Creating Triggers and Alerts.
5. Creating Schedules and Profiles
Post Reply
varghesesa
Posts: 31
Joined: Thu Jul 11, 2019 9:52 pm

Camera Stream Optimization

Post by varghesesa » Thu Feb 04, 2021 3:18 pm

** If you are reading this article, we assume you went through all the necessary steps in the "IP Camera Connections" article to connect your cameras. **

Introduction
This article documents the many pitfalls with camera settings and helps you self-correct.

If you prefer to watch the webinar associated with this article, checkout our YouTube channel. Webinar name: Camera Connections and Streams.

Reference: If you want to learn more about how streaming and encoding works, a great reference article was created by IPVM titled "H.264 vs MJPEG - Quality and Bandwidth Tested". If you are not familiar with IPVM, they are a great resource for research and information regarding surveillance and security cameras.

Table of Contents Issue
We receive tickets associated with the camera streams being choppy, lagging in time, stuttering, loses signal or no signal error followed up with, however, the camera is up and working fine in the camera vendor's app.

The issue here is the vendor app is pulling streams from a proprietary port and protocol. The app is also an active solution, i.e. a user opened the app and wants to view a camera so the camera wakes up and delivers a quality, bandwidth optimized stream to the user.

Blue Iris (BI), like all VMS systems, is passively viewing all the time. Because we are a third party solution, we have access to different ports and settings on the available ports are crucial for BI to deliver a quality user experience.

Camera Stats

BI provides two ways to understand the health of your camera streams. For an individual camera, Camera settings -> General tab provides all the relevant information for an individual camera.

camera_stats_optimized.png
camera_stats_optimized.png (228.82 KiB) Viewed 1097 times

Main stream stats:
  • Resolution of camera: 2.1 MP
  • Frames per second (fps): 31.76
  • Key frame interval per second: 0.70 fps
  • Bit rate: 381.9 kB/s
Sub stream stats (if connected):
  • Resolution of camera: 0.3 MP (640 x 480)
  • Frames per second (fps): 29.77
  • Key frame interval per second: 0.60 fps
  • Bit rate: 42.4 kB/s
Frames per second and Key frame interval per second have the most bearing on smooth streams. For most people, an FPS value above 12 fps and a key frame interval above 0.5 and ideally 1.0 are good guidelines for a good streaming experience.

To get a holistic view of the performance of all your cameras, go to Status -> Camera tab.
camera_log_optimized.png
camera_log_optimized.png (59.57 KiB) Viewed 3222 times
  • Name: Name of camera
  • IP: IP Address of cameras
  • MPixels: Camera resolution main stream/sub stream
  • HA: Hardware Acceleration (I = Intel, N = NVidia)
  • Elapsed: Length of time camera has been running
  • FPS/key: frames per second / key frame interval per second for main stream
  • Bitrate: bitrate for main stream
  • Sub FPS/key: frames per second / key frame interval per second for sub stream
  • Sub Bitrate: bitrate for sub stream
Status counters
  • Motion: Motion represents the number of motion events, not necessarily leading to a Trigger or Alert.
  • Trigger: Trigger represents the number of trigger events, when there was sufficient motion to trigger,
    or the camera was triggering in another way. If this cell is black, the camera is currently in
    the triggered state.
  • Alerts: Alerts represents the number of times that a trigger resulted in one or more alerts fired—
    emails, push notifications, alarms etc.
  • Clips: Clips represents the number of files created. If this cell is black, the camera is actively
    recording.
  • Posted represents the number of frames sent via FTP or saved to disc according to settings on
    the Post page in camera settings.
  • Webcast: Webcast represents the number of frames viewed by web server or app users.
  • NoSignal: NoSignal represents the number of times the camera signal was lost. The signal may have
    been immediately restored causing no loss of video, or it may have been out for longer.
    These events are logged to the Log page in status.
Troubleshooting
At this point, you should know what your cameras are doing and where they are performing poorly.

Keep in mind, users often set hardware acceleration on globally and later notice some cameras have switched from Camera settings -> Video -> Hardware decode from Default -> No. BI detects errors decoding your camera streams using your hardware and changes the setting behind the scenes. The steps discussed below may help resolve the issue. Alternatively, driver updates may occasionally resolve the issue. And at times, the only solution is to leave the setting at No and let BI decode the camera stream via software. As long as the CPU utilization is reasonable, the No setting should be fine.


Video Pipeline: Camera settings x BI Settings

In order for BI and your cameras to work well together, you have to take into account both settings.

Camera settings

Below are the most common camera settings that can affect the video quality seen in Blue Iris. Think of these settings as levers on the camera (NOT BI) that can be adjusted to provide the optimal video stream.
  • H.264 vs H.265
  • H.264/265 Encoding Profiles: Baseline, Main, High etc
  • Key frame interval
  • FPS
Video pipelines are discussed below. Just keep in mind, all the settings above affect the "Camera video" node.

BI only works with the H.264 and H.265 standards. Camera manufacturers have tweaked their encoding to optimize bandwidth for their apps. H.264+, H.265+ (note the + sign) Smart Codec etc are all proprietary encodings that can only be used by the vendor apps. When customers report the cameras work on their app but not in BI, the encoding is often times the reason. The No Signal Error is almost always due to the fact that the cameras are using a proprietary encoding.

Also best to start with H.264 encoding before moving to H.265. Also, choose H.264 Main/Baseline instead of High if given the choice. Provide BI the easiest connection available first and increase the complexity if and when needed. Finally, it just may not be possible for your hardware video acceleration (hwva) to handle the camera's encoding perhaps due to old drivers. In that case, you may just need to turn hwva off for a particular camera.

BI Settings

Below is the list of BI settings that can affect streaming.
  • Hardware decode (aka hardware video acceleration, HWVA).
    Used to turn HA on/off for live camera feeds.
    • Global settings -> Cameras tab. Turn on/off globally for All cameras.
    • Camera settings -> Video tab. Turn on/off for a particular camera.
      Camera setting takes precedence over global setting.
  • Limit decoding unless required. Camera settings -> Video tab.
    Used on the Live view stream. However, largely not needed anymore with the popularity of dual streams.
    A few Gotchas below are associated with this setting.
  • Also BVR. Camera settings -> Video tab.
    Used to turn HA on/off for video playback.
  • Recording settings
    • D2D. Camera settings -> Record tab -> Video file format and compression... Used to determine format of recordings.
      Saves content directly from the camera to the file.
    • Re-encode. Camera settings -> Record tab -> Video file format and compression... Used to determine format of recordings.
      BI processes the camera stream and re-encodes in H.264 format before saving to file.
  • Pre-trigger video buffer. Camera settings -> Record tab.
    Largely set based on user preference. However, can result in video gotchas on certain occasions.
Video Pipeline: Camera settings x BI Settings

All the Gotchas and fixes to gotchas below are based on the combination of Camera settings x BI settings. In order to really understand the fix, understanding the video pipelines that exist in BI is pertinent. Once you understand the pipelines, it becomes much easier to know what effect the Camera settings x BI settings have on the pipeline. If you are curious to go deeper, reference the Blue Iris Streaming Overview article.


Image

*** Coming soon: Direct to wire ***
Bypass decoding / encoding completely. Send the camera encoding straight to remote endpoints. The remote device (mobile app, UI3, web site, YouTube etc) is responsible for decoding the stream successfully.



Gotchas: Video Artifacts

Examples of video distortions

Gotcha 1: Console - Live view

Reported Issues:
  • Green bars as seen below.
    Encoding-HighToBaseline.png
    Encoding-HighToBaseline.png (52.99 KiB) Viewed 263 times
  • Washed out image
    console-live-view-artifact.png
    console-live-view-artifact.png (95.3 KiB) Viewed 263 times
The fix:

Video path: Camera video/feed -> Decode -> Console view
Camera feed: Simplify encoding. In this case, we changed the camera's encoding from H.264 High to H.264 Baseline. Alter Camera encoding settings until streams work.

Alternative possibilities: If camera settings did not resolve the issue, the next step would be to adjust the Decode settings.
Decode: Sometimes your hardware just cannot handle the camera's encoding. In this case the only option is to turn hwva off and decode the cameras via BI software.
Set Hardware decode = No in Camera settings -> Video tab.

Best practice is to always check Status -> Log for any errors.


Gotcha 2: Console - Live view

Reported Issue:
Ghost images. See below.
ghost.png
ghost.png (174.73 KiB) Viewed 716 times

Same issue as Gotcha 1, but I liked the ghost images so decided to share. :)

Ghost images occur when BI cannot decode the intermediate frames (p-frames / b-frames) coming from the camera. The image on the left, i.e. the actual scene was in the same mp4 video (video created on phone recording the console, not an mp4 export. to be discussed below) and would flash every 1s which was the key frame. In the ghost image (intermediate frame) you can kind of make out the mailbox behind the man walking.
I focused in on the ghost image in the second image so you could see the correlation to the original scene. The image is not to scale.

The fix:

Video path: Camera video / feed -> Decode -> Console view
  • Decode: Turned off hwva.
    In this case, the encoding/decoding incompatibility was between the camera and Intel QuickSync. For some reason, the camera stats were saying the key frame ratio was 30 (mjpeg stream). It was almost as if the H.265 encoding from the camera was interpreted by the hwva as an mjpeg stream and thus rendering each intermediate frame as if it were a key frame. When hwva was turned off, the key frame ratio was 1 just as the camera settings show (GOP 1 I-frame / 1s).
As mentioned earlier, the user used his mobile device to create the video, not exporting to a file in BI.
The user stated when exporting to a file (MP4), the video was fine. Big clue.
The video path for exporting to video is: Read the file (recording) -> Decode -> Encode -> Save to MP4.
I also know the default export setting is to re-encode using BI software (i.e. skip hwva).
Thus hwva is the only difference between the two video paths, so hwva must be causing the issue.


Gotcha 3: Export to file

Pulsating jaggies happening at consistent intervals, e.g. every 1s.


pulsating jaggies.png
pulsating jaggies.png (200.56 KiB) Viewed 721 times
Reported issue:
The user was attaching a 10s MP4 file with their email alerts.

Symptom: The video starts out fine but consistently every 1s or 2s, i.e. key frame interval, the image gets very pixelated. The key symptom is the fact this happens in a consistent interval like every 1s or 2s implying BI or the stream is having issues processing the key frame.

The Fix:

Video path: Recorded file -> Decode -> Encode -> Save to file (MP4)
  • Encode: Turn off hwva.
    We got lucky. As you can see, the video path has many hops that could cause issues.
    Starting from the end and working backwards is always a good strategy.


Gotcha 4: Mobile device - Playback

Reported issues:
  • Playback is a gray screen. Playback is a black screen.
  • Randomly restarts from the beginning. Playback hangs.
Video Pipeline: Recorded file -> Decode -> Encode -> Mobile device

Fix: Simplified encoding settings. Global settings -> Web server -> Advanced -> Configure (Stream 0)
Initial guess which turned out to be true was the streaming player on the mobile phone could not handle the stream coming from BI. Below are settings that usually work. Most important settings:
  • Hardware acceleration = No. Critical fix.
  • Maybe limit bit rate will have an effect.
  • Maybe Resize output frame width x height will have an effect.
encoder options.png
encoder options.png (44.06 KiB) Viewed 282 times

If you know the video pipeline, you can now strategically alter different steps to get resolution. If the issue remained, alternative suggestions:
  • Turn off Also BVR. Camera settings -> Video tab. This will cause playback to be decoded using software. (Decode node in video path)
  • If camera is recording D2D (Recorded file node in video path), maybe the camera's encoding is too complicated for the mobile device.
    Simplify the camera's encoding settings. e.g. H.265 -> H.264, High -> Main or Baseline.
  • Switch from D2D to Re-encode. Simplify the recording file (Recorded file node in video path)

Gotchas: Streaming Artifacts


Examples of Stream quality issues


Gotcha 5: Console - Live view

Reported issue: Choppy streams
Choppy streams, i.e. the live view or the playback is not smooth or the seconds on the time overlay are not sequential. These issues are almost entirely due to camera encoding settings. If the cameras are speaking Spanish and BI is speaking English, neither party can understand each other resulting in a poor experience.

The Fix:

Video path: Camera video -> Decode -> Console view
  • Camera video: Check out the camera stats, Status -> Cameras tab. The FPS should be at least 15 fps.
  • Camera video: Look at the key frame ratio in Camera -> Stats. Anything less than 0.5 may provide a bad experience. Set Key frame interval = fps, e.g. 15.

Gotcha 6: Console - Live view

Reported issue: Camera streams are lagging or sluggish or high latency on the console

Fix:

Video path: Camera video -> Decode -> Console view

Almost always, camera streams are sluggish because BI is not receiving enough key frames (i-frames). Need to alter encoding settings on the camera (Camera video node in video path)

As stated above, key frame ratios less than 0.5 lead to sluggish cameras.

Rule of thumb: Make sure the key frame (i-frame) interval on your camera equals the FPS for the camera. The only reason you would consider increasing the interval is if network bandwidth is becoming an issue.

Furthermore, keep settings simple such that all the streams have the same FPS. Choosing between 30 fps or 15 fps is a user preference. Setting all streams to have the same fps makes time reconciliation easier for BI. Turns out certain vendors like Hikvision set different fps (10, 15) for different streams. Different fps settings unnecessarily complicate multiple streams for BI.

Examples:
  • Camera fps = 30. key frame interval = 30 (1.0 fps). Can consider key frame interval as high as = 60 (0.5 fps)
  • Camera fps = 15. key frame interval = 15 (1.0 fps). Can consider key frame interval as high as = 30 (0.5 fps)
Every camera vendor has a different UI. However, observe the default settings for the i-frame interval for each of the streams for the camera vendor (below). All the streams are 15 fps. However the i-frame interval is 30/60/120! The only setting that may provide an ok user experience is the main stream and most users will record D2D for the main stream. If dual streams are connected, the live view will probably be choppy, slow, sluggish! The i-frame interval for the two streams chosen to be used by BI for below example needs to be lowered to 15.
key-frame-sample_optimized.png
key-frame-sample_optimized.png (80.17 KiB) Viewed 3219 times


CPU maxed out

The other reason for lagging camera feeds in live view is the cameras are high resolution and the BI server cannot keep up. If your CPU is running above 80%, it is working too hard. In my opinion, above 50% is concerning. My CPU with (4) 2 MP cameras (I know, not much) hovers around 8%.

Below are the general steps to tune your system if you want to confirm everything is setup optimally.
  • For all the high res (2MP+) cameras you need to have dual streams. (In regards to the Video path, you are adjusting the Camera video node in the video path)
    The IP Camera Connections article walks through best practices on how to do so.
  • Record direct to disk not re-encode. Saves CPU usage.
  • Turn on hardware acceleration if you have it. (Adjusting the Decode node in the video path). See Streaming Overview article for details.


Gotcha 7: Console - Live view

Reported issue:
  • Stuttering at common intervals, e.g. every 1 or 2s.
The fix:

Video path: Camera video/feed -> Decode -> Console view
  • Camera feed: Change stream setting from Constant Bit Rate (CBR) to Variable Bit Rate (VBR)
    Suppose a key frame is sent from the camera every 1s. If the CBR setting is too low, then the camera needs to perform image optimizations to get the frame across the network. Those optimizations lead to the glitches. So simply changing the camera setting to VBR instead of CBR should resolve the issue. In general, setting your cameras with VBR is a better user experience. CBR should only be considered if you have network bandwidth issues.


Gotcha 8: No signal / Loss of signal

The No Signal error can be due to so many issues, we decided to make it a separate article! See Camera Troubleshooting: No Signal Error / Loss of signal



Next steps / Submitting a Ticket
At this point, if you cannot get a good quality stream, provide the following information:
  • Make / Model of camera.
  • Identify any errors in the Status -> log.
  • A screen shot of your camera stats. Status -> Camera tab.
  • The article mentions many checks to resolve the issue. List out what you tested and results. In particular, what are the video stream settings in your camera (encoding, fps, i-frame interval etc). This gives us better insight as to what may be the issue.
  • Send the .reg file for one of the problematic cameras. (Camera settings -> General tab -> Export button) Not necessary to send all the cameras .reg file. Fixing one often leads to cut/paste fixes for the others. There should be no privacy concerns. Unless the cameras are on the internet and we know the WAN address, camera access is impossible.
  • Put the camera on the internet for remote access/testing. We would be happy to do some remote testing.  Could you please send a WAN address for this camera for testing purposes with necessary ports, usually just 80 and 554 (RTSP) and 8999(ONVIF if available).  Don't forget a temp login as well. If you put the camera on the internet we can take a look.  This video explains how to do so. To be clear, we need direct access to the camera, not the BI web interface. 
Post Reply