We receive tickets associated with the camera streams being choppy, lagging in time, stuttering, loses signal or no signal error followed up with, however, the camera is up and working fine in the camera vendor's app.
The issue here is the vendor app is pulling streams from a proprietary port and protocol. The app is also an active solution, i.e. a user opened the app and wants to view a camera so the camera wakes up and delivers a quality, bandwidth optimized stream to the user.
Blue Iris (BI), like all VMS systems, is passively viewing all the time. Because we are a third party solution, we have access to different ports and settings on those ports are crucial for BI to deliver a quality user experience.
BI provides two ways to understand the health of your camera streams. For an individual camera, Camera settings -> General tab provides all the relevant information.
Main stream stats:
- Resolution of camera: 2.1 MP
- Frames per second (fps): 31.76
- Key frame interval per second: 0.70 fps
- Bit rate: 381.9 kB/s
- Resolution of camera: 0.3 MP (640 x 480)
- Frames per second (fps): 29.77
- Key frame interval per second: 0.60 fps
- Bit rate: 42.4 kB/s
To get a holistic view of the performance of all your cameras, go to Status -> Camera tab.
- Name: Name of camera
- IP: IP Address of cameras
- MPixels: Camera resolution main stream/sub stream
- HA: Hardware Acceleration (I = Intel, N = NVidia)
- Elapsed: Length of time camera has been running
- FPS/key: frames per second / key frame interval per second for main stream
- Bitrate: bitrate for main stream
- Sub FPS/key: frames per second / key frame interval per second for sub stream
- Sub Bitrate: bitrate for sub stream
- Motion: Motion represents the number of motion events, not necessarily leading to a Trigger or Alert.
- Trigger: Trigger represents the number of trigger events, when there was sufficient motion to trigger,
or the camera was triggering in another way. If this cell is black, the camera is currently in
the triggered state.
- Alerts: Alerts represents the number of times that a trigger resulted in one or more alerts fired—
emails, push notifications, alarms etc.
- Clips: Clips represents the number of files created. If this cell is black, the camera is actively
- Posted represents the number of frames sent via FTP or saved to disc according to settings on
the Post page in camera settings.
- Webcast: Webcast represents the number of frames viewed by web server or app users.
- NoSignal: NoSignal represents the number of times the camera signal was lost. The signal may have
been immediately restored causing no loss of video, or it may have been out for longer.
These events are logged to the Log page in status.
At this point, you should know what your cameras are doing and where they are performing poorly.
Choppy streams are almost entirely due to camera encoding settings. If the cameras are speaking Spanish and BI is speaking English, neither party can understand each other resulting in a poor experience.
BI only works with the H.264 and H.265 standards. Camera manufacturers have tweaked their encoding to optimize bandwidth for their apps. H.264+, H.265+ (note the + sign) Smart Codec etc are all proprietary encodings that can only be used by the vendor apps. When customers report the cameras work on their app but not in BI, the encoding is often times the reason. The No Signal Error is almost always due to the fact that the cameras are using a proprietary encoding.
Also best to start with H.264 encoding first before moving to H.265. Also, choose H.264 Main/Baseline instead of High if given the choice.
Below is a screenshot from a bvr playback of a camera that was encoding H.264 High direct to disk. The green/gray bars shows the possible distortion when BI cannot understand the camera's encoding.
Key frames are the entire image, mostly unencoded. For a better understanding of key frames, when playing back the above bvr, every 2s an almost correct image would appear. From the camera stats, I realized the key frame interval was 0.5 fps, i.e. a key frame every 2s. Thus, the image must have been the BI software re-encoding the key frame.
The actual image when BI can understand the encoding. In this case, we changed the camera's encoding from H.264 High to H.264 Baseline.
Camera streams are lagging or sluggish
Almost always, camera streams are sluggish because BI is not receiving enough key frames (i-frames). As stated above, key frame ratios less than 0.5 lead to sluggish cameras.
Rule of thumb: Make sure the key frame (i-frame) interval on your camera equals the FPS for the camera. The only reason you would consider increasing the interval is if network bandwidth is becoming an issue.
Furthermore, keep settings simple such that all the streams have the same FPS. Choosing between 30 fps or 15 fps is a user preference.
- Camera fps = 30. key frame interval = 30 (1.0 fps). Can consider key frame interval as high as = 60 (0.5 fps)
- Camera fps = 15. key frame interval = 15 (1.0 fps). Can consider key frame interval as high as = 30 (0.5 fps)
Cameras are stuttering or routinely glitching
Another occasional issue that pops up is the stream glitches in a very consistent interval like 2s. This is almost always due to the camera being set to Constant Bit Rate (CBR) instead of Variable Bit Rate (VBR). Many cameras have a default key frame rate set to 0.5 which means a new key frame (full frame) is sent every 2s. If the CBR setting is too low, then the camera needs to perform image optimizations to get the frame across the network. Those optimizations lead to the glitches. So simply changing the camera setting to VBR instead of CBR should resolve the issue. In general, setting your cameras with VBR is a better user experience. CBR should only be considered if you have network bandwidth issues.
Loss of signal
Camera settings -> Watchdog allows you to set alerts when a camera losses signal. This is a good tool to understand the scope of the problem and investigate the issue at the time it occurs.
Best way to address lost signal is by first looking at the status -> camera stats as mentioned above. Is the FPS > 12 fps? If not, the camera is not sending enough frames to BI. This could be due to the network, the network settings or the ethernet card on the BI server. Troubleshooting network issues is beyond BI support.
A good test for lost signal is to connect to the camera via the vendor's phone app. If the BI connection re-establishes as soon as the vendor's phone app connects to the camera, this is a good indication that the encoding setting on the camera to BI is not following the H.264/H.265 standards as mentioned earlier. The camera app is re-establishing a "live" connection to the camera, basically waking the camera up.
Is the key frame interval < 0.5? Then BI is not getting enough key frames as stated above. Key frame interval is a camera setting. Key frames are also affected by camera encoding settings. Again, BI only supports H.264 (baseline preferably) or H.265 encoding. As mentioned earlier, other encodings lead to lag, sluggishness or loss of signal.
Loss of signal could also be due to maybe the camera being overloaded. If you are pulling dual streams, does the fps go up if BI only pulls one stream? If you can pull the main stream or the sub stream correctly, but not both together, you may need to reach out to the camera vendor for more information.
Finally the hardware acceleration can lead to loss of signal. The camera encoding may not be compatible with the hardware acceleration. A simple test is to turn off HA for the cameras with issues and see if they start working again.
If many cameras (from same vendor) suddenly show loss of signal after a BI update, the issue could just be coincidental with the BI update. We have experienced tickets where users reported camera loss of signal after a BI update which coincidentally, also had a Windows update. After investigating, we realized the Windows update had affected the Intel drivers resulting in cameras that worked before, no longer working. Turning off HA for those cameras allowed us to discover the core issue.
Finally, when there is a loss signal, check Status -> logs to see if BI reports any errors.
RTMP vs RTSP Protocol
99% of cameras only support the RTSP protocol. On occasion, a vendor will support RTSP and RTMP. Reolink is one such vendor and the RTMP protocol seems to provide a healthier stream.
Checkout the Reolink Gotchas article for best practices on connecting Reolink cameras.
BI Support has also been compiling and sharing other gotchas associated with camera settings which we have uncovered through trouble-shooting with customers.
At this point, you should have good camera connectivity. Now you are ready to optimize your server further.
Record Direct to Disk
Camera settings -> Record tab -> Video file format and compression -> Direct-to-disc should be selected. Keep in mind, if you have configured dual streams, D2D is the only way to store both streams. If the re-encode option is selected with dual streams, the sub stream will only be stored. When playing back clips or alerts, you will notice this distinction.
Confirm BI still works well during playback of clips and / or alerts.
Hardware acceleration can be turned on/off globally. Global settings -> Cameras -> Hardware accelerated decode
Hardware acceleration can also be turned on/off at the camera level. Or if you have an Intel CPU with a NVidia graphics, for example, you could mix and match which cameras use which hardware to balance the load. At the camera level, the setting is Camera settings -> Video tab -> Hardware decode. As mentioned above, the Status -> Camera tab will display which hardware is being used by which camera.
With dual streams and hardware acceleration, the need for "Limit decoding unless required" and "Also BVR" are largely no longer needed. They remain in the software largely for backwards compatibility for those users that have not upgraded their hardware or updated their software.
Some final thoughts
This article progressively moves you through the steps needed to optimize camera connections.
Camera settings -> D2D Recordings -> Hardware acceleration.
When troubleshooting, work backwards, simplify the problem and see if the streams improve. By working backwards, you can identify the issue.
Turn off hardware acceleration -> Switch recording back to re-encode -> Adjust camera settings
Turn off hardware acceleration. Do the streams improve?
Switch back to Re-encode in recording settings. Do the streams improve?
Submitting a Ticket
At this point, if you cannot get a good quality stream, provide the following information:
- Identify any errors in the Status -> log.
- If the problem is with a single camera, send a screen shot of the camera stats for that particular camera. (Camera settings -> General tab) If the problem is with many cameras, send a screen shot of the camera stats. (Status -> Camera tab)
- Send the .reg file for one of the problematic cameras. (Camera settings -> General tab -> Export button) Not necessary to send all the cameras .reg file. Fixing one often leads to cut/paste fixes for the others. There should be no privacy concerns. Unless the cameras are on the internet and we know the WAN address, camera access is impossible.
- Easiest way to troubleshoot is to put the camera on the internet for remote access. We would be happy to do some remote testing. Could you please send a WAN address for this camera for testing purposes with necessary ports, usually just 80 and 554 (RTSP) and 8999(ONVIF if available). Don't forget a temp login as well. If you put the camera on the internet we can take a look. This video explains how to do so. To be clear, we need direct access to the camera, not the BI web interface.