Camera Stream Optimization

Details on the five major steps to configure and deploy Blue Iris.
1. Connecting IP cameras.
2. Configuring your home or office network for Remote Access via a web browser or the phone apps.
3. Setting up Storage and Recordings.
4. Creating Triggers and Alerts.
5. Creating Schedules and Profiles
Post Reply
varghesesa
Posts: 28
Joined: Thu Jul 11, 2019 9:52 pm

Camera Stream Optimization

Post by varghesesa » Thu Feb 04, 2021 3:18 pm

** If you are reading this article, we assume you went through all the necessary steps in the "IP Camera Connections" article to connect your cameras. **

Introduction
This article documents the many pitfalls with camera settings and helps you self-correct.

If you prefer to watch the webinar associated with this article, checkout our YouTube channel. Webinar name: Camera Connections and Streams.

Reference: If you want to learn more about how streaming and encoding works, a great reference article was created by IPVM titled "H.264 vs MJPEG - Quality and Bandwidth Tested". If you are not familiar with IPVM, they are a great resource for research and information regarding surveillance and security cameras.

Table of Contents Issue
We receive tickets associated with the camera streams being choppy, lagging in time, stuttering, loses signal or no signal error followed up with, however, the camera is up and working fine in the camera vendor's app.

The issue here is the vendor app is pulling streams from a proprietary port and protocol. The app is also an active solution, i.e. a user opened the app and wants to view a camera so the camera wakes up and delivers a quality, bandwidth optimized stream to the user.

Blue Iris (BI), like all VMS systems, is passively viewing all the time. Because we are a third party solution, we have access to different ports and settings on the available ports are crucial for BI to deliver a quality user experience.

Camera Stats

BI provides two ways to understand the health of your camera streams. For an individual camera, Camera settings -> General tab provides all the relevant information for an individual camera.

camera_stats_optimized.png
camera_stats_optimized.png (228.82 KiB) Viewed 1042 times

Main stream stats:
  • Resolution of camera: 2.1 MP
  • Frames per second (fps): 31.76
  • Key frame interval per second: 0.70 fps
  • Bit rate: 381.9 kB/s
Sub stream stats (if connected):
  • Resolution of camera: 0.3 MP (640 x 480)
  • Frames per second (fps): 29.77
  • Key frame interval per second: 0.60 fps
  • Bit rate: 42.4 kB/s
Frames per second and Key frame interval per second have the most bearing on smooth streams. For most people, an FPS value above 12 fps and a key frame interval above 0.5 and ideally 1.0 are good guidelines for a good streaming experience.

To get a holistic view of the performance of all your cameras, go to Status -> Camera tab.
camera_log_optimized.png
camera_log_optimized.png (59.57 KiB) Viewed 3167 times
  • Name: Name of camera
  • IP: IP Address of cameras
  • MPixels: Camera resolution main stream/sub stream
  • HA: Hardware Acceleration (I = Intel, N = NVidia)
  • Elapsed: Length of time camera has been running
  • FPS/key: frames per second / key frame interval per second for main stream
  • Bitrate: bitrate for main stream
  • Sub FPS/key: frames per second / key frame interval per second for sub stream
  • Sub Bitrate: bitrate for sub stream
Status counters
  • Motion: Motion represents the number of motion events, not necessarily leading to a Trigger or Alert.
  • Trigger: Trigger represents the number of trigger events, when there was sufficient motion to trigger,
    or the camera was triggering in another way. If this cell is black, the camera is currently in
    the triggered state.
  • Alerts: Alerts represents the number of times that a trigger resulted in one or more alerts fired—
    emails, push notifications, alarms etc.
  • Clips: Clips represents the number of files created. If this cell is black, the camera is actively
    recording.
  • Posted represents the number of frames sent via FTP or saved to disc according to settings on
    the Post page in camera settings.
  • Webcast: Webcast represents the number of frames viewed by web server or app users.
  • NoSignal: NoSignal represents the number of times the camera signal was lost. The signal may have
    been immediately restored causing no loss of video, or it may have been out for longer.
    These events are logged to the Log page in status.
Troubleshooting
At this point, you should know what your cameras are doing and where they are performing poorly.

Keep in mind, users often set hardware acceleration on globally and later notice some cameras have switched from Camera settings -> Video -> Hardware decode from Default -> No. BI detects errors decoding your camera streams using your hardware and changes the setting behind the scenes. The steps discussed below may help resolve the issue. Alternatively, driver updates may occasionally resolve the issue. And at times, the only solution is to leave the setting at No and let BI decode the camera stream via software. As long as the CPU utilization is reasonable, the No setting should be fine.


Video Pipeline: Camera settings x BI Settings

In order for BI and your cameras to work well together, you have to take into account both settings.

Camera settings

Below are the most common camera settings that can affect the video quality seen in Blue Iris. Think of these settings as levers on the camera (NOT BI) that can be adjusted to provide the optimal video stream.
  • H.264 vs H.265
  • H.264/265 Encoding Profiles: Baseline, Main, High etc
  • Key frame interval
  • FPS
Video pipelines are discussed below. Just keep in mind, all the settings above affect the "Camera video" node.

BI only works with the H.264 and H.265 standards. Camera manufacturers have tweaked their encoding to optimize bandwidth for their apps. H.264+, H.265+ (note the + sign) Smart Codec etc are all proprietary encodings that can only be used by the vendor apps. When customers report the cameras work on their app but not in BI, the encoding is often times the reason. The No Signal Error is almost always due to the fact that the cameras are using a proprietary encoding.

Also best to start with H.264 encoding before moving to H.265. Also, choose H.264 Main/Baseline instead of High if given the choice. Provide BI the easiest connection available first and increase the complexity if and when needed. Finally, it just may not be possible for your hardware video acceleration (hwva) to handle the camera's encoding perhaps due to old drivers. In that case, you may just need to turn hwva off for a particular camera.

BI Settings

Below is the list of BI settings that can affect streaming.
  • Hardware decode (aka hardware video acceleration, HWVA).
    Used to turn HA on/off for live camera feeds.
    • Global settings -> Cameras tab. Turn on/off globally for All cameras.
    • Camera settings -> Video tab. Turn on/off for a particular camera.
      Camera setting takes precedence over global setting.
  • Limit decoding unless required. Camera settings -> Video tab.
    Used on the Live view stream. However, largely not needed anymore with the popularity of dual streams.
    A few Gotchas below are associated with this setting.
  • Also BVR. Camera settings -> Video tab.
    Used to turn HA on/off for video playback.
  • Recording settings
    • D2D. Camera settings -> Record tab -> Video file format and compression... Used to determine format of recordings.
      Saves content directly from the camera to the file.
    • Re-encode. Camera settings -> Record tab -> Video file format and compression... Used to determine format of recordings.
      BI processes the camera stream and re-encodes in H.264 format before saving to file.
  • Pre-trigger video buffer. Camera settings -> Record tab.
    Largely set based on user preference. However, can result in video gotchas on certain occasions.
Video Pipeline: Camera settings x BI Settings

All the Gotchas and fixes to gotchas below are based on the combination of Camera settings x BI settings. In order to really understand the fix, understanding the video pipelines that exist in BI is pertinent. Once you understand the pipelines, it becomes much easier to know what effect the Camera settings x BI settings have on the pipeline. If you are curious to go deeper, reference the Blue Iris Streaming Overview article.


Image

*** Coming soon: Direct to wire ***
Bypass decoding / encoding completely. Send the camera encoding straight to remote endpoints. The remote device (mobile app, UI3, web site, YouTube etc) is responsible for decoding the stream successfully.



Gotchas: Video Artifacts

Examples of video distortions

Gotcha 1: Console - Live view

Reported Issues:
  • Green bars as seen below.
    Encoding-HighToBaseline.png
    Encoding-HighToBaseline.png (52.99 KiB) Viewed 208 times
  • Washed out image
    console-live-view-artifact.png
    console-live-view-artifact.png (95.3 KiB) Viewed 208 times
The fix:

Video path: Camera video/feed -> Decode -> Console view
Camera feed: Simplify encoding. In this case, we changed the camera's encoding from H.264 High to H.264 Baseline. Alter Camera encoding settings until streams work.

Alternative possibilities: If camera settings did not resolve the issue, the next step would be to adjust the Decode settings.
Decode: Sometimes your hardware just cannot handle the camera's encoding. In this case the only option is to turn hwva off and decode the cameras via BI software.
Set Hardware decode = No in Camera settings -> Video tab.

Best practice is to always check Status -> Log for any errors.


Gotcha 2: Console - Live view

Reported Issue:
Ghost images. See below.
ghost.png
ghost.png (174.73 KiB) Viewed 661 times

Same issue as Gotcha 1, but I liked the ghost images so decided to share. :)

Ghost images occur when BI cannot decode the intermediate frames (p-frames / b-frames) coming from the camera. The image on the left, i.e. the actual scene was in the same mp4 video (video created on phone recording the console, not an mp4 export. to be discussed below) and would flash every 1s which was the key frame. In the ghost image (intermediate frame) you can kind of make out the mailbox behind the man walking.
I focused in on the ghost image in the second image so you could see the correlation to the original scene. The image is not to scale.

The fix:

Video path: Camera video / feed -> Decode -> Console view
  • Decode: Turned off hwva.
    In this case, the encoding/decoding incompatibility was between the camera and Intel QuickSync. For some reason, the camera stats were saying the key frame ratio was 30 (mjpeg stream). It was almost as if the H.265 encoding from the camera was interpreted by the hwva as an mjpeg stream and thus rendering each intermediate frame as if it were a key frame. When hwva was turned off, the key frame ratio was 1 just as the camera settings show (GOP 1 I-frame / 1s).
As mentioned earlier, the user used his mobile device to create the video, not exporting to a file in BI.
The user stated when exporting to a file (MP4), the video was fine. Big clue.
The video path for exporting to video is: Read the file (recording) -> Decode -> Encode -> Save to MP4.
I also know the default export setting is to re-encode using BI software (i.e. skip hwva).
Thus hwva is the only difference between the two video paths, so hwva must be causing the issue.


Gotcha 3: Export to file

Pulsating jaggies happening at consistent intervals, e.g. every 1s.


pulsating jaggies.png
pulsating jaggies.png (200.56 KiB) Viewed 666 times
Reported issue:
The user was attaching a 10s MP4 file with their email alerts.

Symptom: The video starts out fine but consistently every 1s or 2s, i.e. key frame interval, the image gets very pixelated. The key symptom is the fact this happens in a consistent interval like every 1s or 2s implying BI or the stream is having issues processing the key frame.

The Fix:

Video path: Recorded file -> Decode -> Encode -> Save to file (MP4)
  • Encode: Turn off hwva.
    We got lucky. As you can see, the video path has many hops that could cause issues.
    Starting from the end and working backwards is always a good strategy.


Gotcha 4: Mobile device - Playback

Reported issues:
  • Playback is a gray screen. Playback is a black screen.
  • Randomly restarts from the beginning. Playback hangs.
Video Pipeline: Recorded file -> Decode -> Encode -> Mobile device

Fix: Simplified encoding settings. Global settings -> Web server -> Advanced -> Configure (Stream 0)
Initial guess which turned out to be true was the streaming player on the mobile phone could not handle the stream coming from BI. Below are settings that usually work. Most important settings:
  • Hardware acceleration = No. Critical fix.
  • Maybe limit bit rate will have an effect.
  • Maybe Resize output frame width x height will have an effect.
encoder options.png
encoder options.png (44.06 KiB) Viewed 227 times

If you know the video pipeline, you can now strategically alter different steps to get resolution. If the issue remained, alternative suggestions:
  • Turn off Also BVR. Camera settings -> Video tab. This will cause playback to be decoded using software. (Decode node in video path)
  • If camera is recording D2D (Recorded file node in video path), maybe the camera's encoding is too complicated for the mobile device.
    Simplify the camera's encoding settings. e.g. H.265 -> H.264, High -> Main or Baseline.
  • Switch from D2D to Re-encode. Simplify the recording file (Recorded file node in video path)

Gotchas: Streaming Artifacts


Examples of Stream quality issues


Gotcha 5: Console - Live view

Reported issue: Choppy streams
Choppy streams, i.e. the live view or the playback is not smooth or the seconds on the time overlay are not sequential. These issues are almost entirely due to camera encoding settings. If the cameras are speaking Spanish and BI is speaking English, neither party can understand each other resulting in a poor experience.

The Fix:

Video path: Camera video -> Decode -> Console view
  • Camera video: Check out the camera stats, Status -> Cameras tab. The FPS should be at least 15 fps.
  • Camera video: Look at the key frame ratio in Camera -> Stats. Anything less than 0.5 may provide a bad experience. Set Key frame interval = fps, e.g. 15.

Gotcha 6: Console - Live view

Reported issue: Camera streams are lagging or sluggish or high latency on the console

Fix:

Video path: Camera video -> Decode -> Console view

Almost always, camera streams are sluggish because BI is not receiving enough key frames (i-frames). Need to alter encoding settings on the camera (Camera video node in video path)

As stated above, key frame ratios less than 0.5 lead to sluggish cameras.

Rule of thumb: Make sure the key frame (i-frame) interval on your camera equals the FPS for the camera. The only reason you would consider increasing the interval is if network bandwidth is becoming an issue.

Furthermore, keep settings simple such that all the streams have the same FPS. Choosing between 30 fps or 15 fps is a user preference. Setting all streams to have the same fps makes time reconciliation easier for BI. Turns out certain vendors like Hikvision set different fps (10, 15) for different streams. Different fps settings unnecessarily complicate multiple streams for BI.

Examples:
  • Camera fps = 30. key frame interval = 30 (1.0 fps). Can consider key frame interval as high as = 60 (0.5 fps)
  • Camera fps = 15. key frame interval = 15 (1.0 fps). Can consider key frame interval as high as = 30 (0.5 fps)
Every camera vendor has a different UI. However, observe the default settings for the i-frame interval for each of the streams for the camera vendor (below). All the streams are 15 fps. However the i-frame interval is 30/60/120! The only setting that may provide an ok user experience is the main stream and most users will record D2D for the main stream. If dual streams are connected, the live view will probably be choppy, slow, sluggish! The i-frame interval for the two streams chosen to be used by BI for below example needs to be lowered to 15.
key-frame-sample_optimized.png
key-frame-sample_optimized.png (80.17 KiB) Viewed 3164 times


CPU maxed out

The other reason for lagging camera feeds in live view is the cameras are high resolution and the BI server cannot keep up. If your CPU is running above 80%, it is working too hard. In my opinion, above 50% is concerning. My CPU with (4) 2 MP cameras (I know, not much) hovers around 8%.

Below are the general steps to tune your system if you want to confirm everything is setup optimally.
  • For all the high res (2MP+) cameras you need to have dual streams. (In regards to the Video path, you are adjusting the Camera video node in the video path)
    The IP Camera Connections article walks through best practices on how to do so.
  • Record direct to disk not re-encode. Saves CPU usage.
  • Turn on hardware acceleration if you have it. (Adjusting the Decode node in the video path). See Streaming Overview article for details.


Gotcha 7: Console - Live view

Reported issue:
  • Stuttering at common intervals, e.g. every 1 or 2s.
The fix:

Video path: Camera video/feed -> Decode -> Console view
  • Camera feed: Change stream setting from Constant Bit Rate (CBR) to Variable Bit Rate (VBR)
    Suppose a key frame is sent from the camera every 1s. If the CBR setting is too low, then the camera needs to perform image optimizations to get the frame across the network. Those optimizations lead to the glitches. So simply changing the camera setting to VBR instead of CBR should resolve the issue. In general, setting your cameras with VBR is a better user experience. CBR should only be considered if you have network bandwidth issues.


Gotcha 8: No signal / Loss of signal

The No Signal error can be due to so many issues, we decided to keep it as a completely separate topic.

The Status -> Camera tab as mentioned above maintains No Signal stats so you can easily get an idea on how big is the problem. Furthermore, Camera settings -> Watchdog allows you to set alerts when a camera losses signal. This is a good tool to understand the scope of the problem and investigate the issue at the time it occurs.

Server power settings

I like to start with the easiest checks first. If you are noticing No Signal issues only when away from the computer, make sure there are no Power saving settings such as Battery saver mode etc. The monitor can go to sleep. The PC needs to be running at full power. Users have reported No signal issues due to power savings mode.

Response from a user with incorrect power savings and getting No signal from his USB camera. Issue applies to USB & IP cameras.
At last, sorted. I said it would probably be my fault and something daft. Well it was, Ken. When I set up the pc I adjusted the power options for the screen to shut down after 10 mins. Also, to sleep after 10 minutes of no activity. This I did not know shuts off the camera even thou the program is running, active. So I did not bother to check that the pc was going to sleep even thou this was running.
Laptops are notorious for having power saving modes that can affect access to system resources such as CPU. In this case, BI was stating 100% CPU usage in the lower right corner of the application. However, the CPU utilization from Windows Task Manager was hovering at 33%! Power savings was throttling BI.
Hey thank you for your help, turns out it is a windows issue not allowing my cpu usage over 40% (capped at 39% as seen on resource monitor). Blue iris is running on a laptop and seems a battery/ac adapter driver/process is malfunctioning. I have not isolated it yet but I confirmed by unplugging and then plugging back in the ac power adapter, now my machine is able to run at 100% cpu when needed. Found this out down a reddit rabbit hole.
Networking

If cameras are reporting No Signal, I would first confirm the network is setup and stable.

Camera App test
A good test for lost signal is to connect to the camera via the vendor's phone app (if available). This confirms the camera is connected to the network correctly.

Observe if the BI connection re-establishes as soon as the vendor's phone app connects to the camera, this is a good indication that the encoding setting on the camera to BI is not following the H.264/H.265 standards. The camera app is re-establishing a "live" connection to the camera, basically waking the camera up.

If the camera comes with a browser interface, login to the camera from a browser. Now you have confirmed the correct IP Address and Username/Password for the camera. FYI, the link is automatically created in BI if you already tried adding the camera. Camera settings -> Video tab. If the camera has a web interface, it's a good sign the camera is accessible by third party software.

camera ip.png
camera ip.png (19.74 KiB) Viewed 60 times


Third Party Apps
The easiest way to confirm the network is working is by seeing if the cameras connect with another 3rd party application.
Download the VLC player or another 3rd party player to your BI machine and connect to the cameras. The VLC article provides details.
Do they continue to play when BI states lost signal? If so, BI settings may be the issue. Try re-adding the camera by walking through IP Camera Connections article.

If another 3rd party player cannot access the camera, then either the network is not setup correctly or the camera/device does not allow 3rd party access. The Remote Access section in the BI Checklist shares all the content available to troubleshoot your network.

Network issues
Many customers have a hard time believing their networking hardware is bad or incorrectly configured. Below are a list of issues to provide clues on how to investigate your network.
  • Router & cables: If some cameras work and others do not work, switch the router ports for the non-working cameras with working cameras. Rule out the router has gone bad. Switch out ethernet cables of working and non-working cameras. Rule out the ethernet cables have not gone bad.
  • I had a user with 5 cameras, most of which were high resolution (4 MP+). The issue was the router which was eventually replaced. The problem was not obvious. 4 of the cameras worked for the most part (but they too would lose signal on occasion). The 8 MP camera (PoE) was the most problematic. Even more strange the lower resolution wireless cameras worked better than the 8 MP camera. The user had no issues after replacing his router. Troubleshooting network issues is beyond BI support.
    No, only one camera was initially causing trouble. I have 4 reolink and the amcrest. Interestingly, the wireless reolink worked great. The poe 810A was the most problematic,..and I think also the highest resolution. Hence, perhaps some timeouts or something with my old / bad modem. No problems since I replace the modem.
Camera hardware
Loss of signal could also be due to maybe the camera being overloaded. If you are pulling dual streams, does the fps go up if BI only pulls one stream? If you can pull the main stream or the sub stream correctly, but not both together, you may need to reach out to the camera vendor for more information.


*** Camera, Network equipment and BI server have been ruled out. Dig into the BI Settings ***

IP Config Dialog

RTMP vs RTSP Protocol: 99% of cameras only support the RTSP protocol. On occasion, a vendor will support RTSP and RTMP. Reolink is one such vendor and the RTMP protocol seems to provide a healthier stream.
Checkout the Reolink Gotchas article for best practices on connecting Reolink cameras.

Camera gotchas: BI Support has also been compiling and sharing other gotchas associated with camera settings which we have uncovered through trouble-shooting with customers.



Video path: Camera video/feed -> Decode -> Console view (fairly simple path)

Video path: Camera video/feed
Set camera to simplest encoding possible, e.g. H.264 Baseline or Main.

Video path: Decode

GPU / Hardware Incompatibility
There is nothing more frustrating than doing a BI update and all your camera streams stop. Many users state the BI Update caused the issue. Sharing this story because it's easy to place fault. The below was a real ticket. :) The customer was convinced the issue was with the BI update until proven otherwise.

The moral to the story is understanding the video path really helps get to the root cause. The second moral is sharing knowledge from past tickets really helps future users.

no-signal_optimized.png
no-signal_optimized.png (65.33 KiB) Viewed 1039 times

Rolling back to the previous version or last stable version did not resolve the issue. The customer was convinced the issue was with the BI update. Instead of trouble-shooting, he waited for the next update, hoping for a fix.

Unfortunately, no fix came. The customer then revisited my trouble-shooting email a few weeks later. I first asked the customer to turn off cameras and to observe whether other cameras started working. I suspected the CPU was being overworked. No improvement after disabling all the cameras but one.

experiment-1_optimized.png
experiment-1_optimized.png (57.58 KiB) Viewed 1036 times

My next suggestion was to turn off hardware acceleration. After doing so for the non-working cameras, all came back online!
experiment-2_optimized.png
experiment-2_optimized.png (141.94 KiB) Viewed 1039 times
We suspect the root cause was probably a Windows update or GPU update that disrupted the camera streams. Moral to the story is BI is dependent on many external drivers and software. All these 3rd party dependencies can lead to instability. Second take away is turn off hardware acceleration if a camera or group of cameras constantly has no signal and see if issue goes away. Hardware acceleration is one of the few settings that can affect many cameras at once.

FYI, as mentioned above, the software now turns off HA automatically for a camera if it determines the stream is not compatible and this is logged to Status->log as well.

hwva.png
hwva.png (28.4 KiB) Viewed 72 times


If you want to fix the problem, you can go to the camera settings on the camera and adjust the encoding to a more simple format that may start working with your hardware. A simple example is if the camera was set to H.264 High encoding, adjust the setting to H.264 Baseline. If you were using H.265, switch to H.264.
Or wait for the next driver update that may fix the issue caused by the current driver.


Final action

If no-signal is still not resolved, narrow down the issue by turning functionality off. Turn the shield from green to red. The red shield turns everything off, e.g. recordings, alerts etc. The cameras remain connected and stream live video. Do the cameras start working again? This could imply the CPU is overloaded when running normally, i.e. storage, remote connections, triggers and alerts etc are enabled.

Temporarily disable all the functioning cameras. Simplify the video pipeline: Camera video/feed -> Decode -> Console view.
  • Camera video/feed (settings on the camera): Change the stream to the simplest encoding possible, e.g. H.264 Baseline or Main.
  • Decode: Turn h/w decode off. Hardware decode = No. Camera settings -> Video tab.
Check Status -> logs to see if BI reports any errors. You may also want to consider deleting and re-adding the camera. I have customer testimonials stating their skepticism to doing so but it has worked on occasion. Some mysteries with Windows and software will never be solved.

Finally, if the cameras worked in the past, go to past versions and see if the cameras connect. If so, include that information with the ticket.





Next steps / Submitting a Ticket
At this point, if you cannot get a good quality stream, provide the following information:
  • Make / Model of camera.
  • Identify any errors in the Status -> log.
  • A screen shot of your camera stats. Status -> Camera tab.
  • The article mentions many checks to resolve the issue. List out what you tested and results. In particular, what are the video stream settings in your camera (encoding, fps, i-frame interval etc). This gives us better insight as to what may be the issue.
  • Send the .reg file for one of the problematic cameras. (Camera settings -> General tab -> Export button) Not necessary to send all the cameras .reg file. Fixing one often leads to cut/paste fixes for the others. There should be no privacy concerns. Unless the cameras are on the internet and we know the WAN address, camera access is impossible.
  • Put the camera on the internet for remote access/testing. We would be happy to do some remote testing.  Could you please send a WAN address for this camera for testing purposes with necessary ports, usually just 80 and 554 (RTSP) and 8999(ONVIF if available).  Don't forget a temp login as well. If you put the camera on the internet we can take a look.  This video explains how to do so. To be clear, we need direct access to the camera, not the BI web interface. 
Post Reply