FPS Drops

General discussion about Blue Iris
Post Reply
Mango99
Posts: 2
Joined: Sun Feb 28, 2021 5:23 pm

FPS Drops

Post by Mango99 »

Hello, I am having a hell of a time getting proper performance from my BI machine. The issue first manifested with moving options getting pixelated and 'going green'. I have fixed this but now get severe FPS drops as a replacement. The FPS drops happen in recordings where the pixelation used to, so I believe this is part of the same underlying issue vs separate problems. I have tried 4 camera models - 2 Annke cameras (1x4K, 1x5MP), a Reolink turret, Amcrest 5MP. I experience the issue with each camera, so it is clearly something with my configuration or server. When the FPS drops, the BI web server properly reflects this - it shows the FPS counter go down. Live view experiences same issues.

PC specs:
Lenovo M910x i7 7700T
24GB RAM
SSDs (BI & OS on one SSD, recordings going to another SSD)
BI v5 purchased (eliminates the direct 2 disk eval issue)

Network specs:
- Cameras run off PoE switch
- Cameras and server on separate VLANs
- Disabled all NGFW related features on the server and camera traffic (previously locked down to specific applications vs ports, etc)
- One way traffic allowed - only server can talk to cameras
- Network gear: Palo Alto firewall, UniFi switch (tried an Aruba switch + PoE injector as well)

I've gone through the optimization doc on IPCamTalk and a number of forum suggestions.
- Matching iFrame to frame rate (both streams)
- Disabling sub stream
- Unchecking 'also BVR'
- Unchecking 'limit decoding unless required'
- Different combinations of HA / no HA, bitrate, VBS, CBS
- Made sure smart codecs disabled
- Direct 2 disk
- Uncheck 'Also BVR'
- Confirmed HA working
- Increasing Receive Buffer
- Setting FPS within BI slightly above the camera FPS for headroom
- Disabling 3dnr / misc camera specific noise reduction & picture quality settings
- Ensuring the 'highlight motion' is disabled
- Tested with 24x7 recordings vs motion trigger (read potential issues with the pre-trigger video buffer)
- Confirmed the FPS/key in BI stats is correct - 15FPS/1.00
- Tried lowering FPS to 10
- Tried different cabling

Next ideas:
- Install BI on a separate PC
- Move cameras to same VLAN as BI server
- Try different NVR or way to record same cameras to isolate issue to BI

I feel the list of remaining ideas involves high effort. I already purchased BI and want to use this PC. Nothing obvious stands out about the hardware, so I wanted to create this post to see if I'm missing something. I wonder if the T model of the i7 7700 makes a difference, it is technically a low power version. However, the CPU usage sits very low when this issue happens. Unless it's a matter of clock speed or some other bottleneck of the T model, I don't see how hardware would be an issue.

Appreciate any suggestions.
User avatar
Thixotropic
Posts: 743
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: FPS Drops

Post by Thixotropic »

This is a long shot, but is there any chance the PoE switch could be causing this?

Maybe try switching ports around to see if it's something related to the ports. It's probably not practical to swap out the switch temporarily but if you could do that it would rule it in or out pretty quickly.

It sounds like you've done just about everything within reason to track it down. Hopefully some of the other people here will have some suggestions.
Blue Iris 5.x x64 | Windows 10 Pro x64 | 16GB RAM | i7-7700 3.6 GHz | 1TB HDD | 2TB RAID NAS | 9 Cameras | Almost Dual NIC | 2KVA UPS
Matts1984
Posts: 496
Joined: Fri Apr 10, 2020 1:12 pm
Location: Maryland, USA

Re: FPS Drops

Post by Matts1984 »

I think I would try the method of putting the server on the same vlan. That SHOULD be minimal cost - just time. You have tried so many options I think the only constant is the vlan hop through the Palo. As strong as my bias is against them, it certainly should be able to handle passing this traffic just fine. That said, physically bypassing it is IMHO the next option.

Based on your description it sounds like you're running the OS and BI directly on the server hardware (versus a VM). Does your hardware have two NICs? Could you simply cable the second NIC to the same UI switch assigned to the camera vlan? Keep your server default gateway and config just as it is and add a second network (with no gateway). The system should then use the second NIC to connect to the cameras, bypassing the firewall. All management and Windows related functionality would continue through the current IP/interface/config. This should help minimize any changes you need to make.

I wasn't particularly having performance issues but this more or less is the way my VM BlueIris server is set up.
Blue Iris 5.8.8.x | Server 2022 VM | Xeon X5650 @ 2.67GHz - 12 Cores | 12GB RAM | 7TB RAID | Sophos UTM WAF | 4x SV3C 5MP Bullet A | 1x SV3C 5MP PTZ HX | 1x SV3C 5MP Bullet HX | 1x SV3C 5MP Dome HX | 2x Amcrest 5MP Bullet
MikeBwca
Posts: 1115
Joined: Thu Jun 20, 2019 5:39 am

Re: FPS Drops

Post by MikeBwca »

Maybe try just 1 camera for several minutes. Then keep adding the other cameras to see if your network is getting saturated. A cell phone data bandwidth is much lower than an internet isp.

Other thoughts...
- Lower the resolutions of the cameras to lower bandwidth.
- Adjust the streaming profile in the webserver.
- Use a lower 'Quality' in the UI3. This dramatically increases the fps & decreases cpu a little for me.
Mango99
Posts: 2
Joined: Sun Feb 28, 2021 5:23 pm

Re: FPS Drops

Post by Mango99 »

I want to thank everyone for their replies.

I made a lot of changes while troubleshooting and am currently running stable, with a single camera at least. Have not added more back yet, but I was having issues with a single camera before. Unfortunately I'm not 100% sure what seemingly fixed it, but now when I disable HA (which I had previously done...) the issue seems gone. I did notice some improvement when HA was enabled after increasing the Receive Buffer, so I think it's a combination of all the misc adjustments with HA now disabled. Which is unfortunate since the CPU supports HA.

Something I did not previously mention - there was a 2nd GPU in the system, which was there when I first tried disabling HA. I don't see how disabling it with vs without the 2nd GPU makes a difference, but that's just another variable in the mix.

So I believe it's fixed, albeit no HA, but time will tell. I'm now going to bang my head against the wall some more as I embark on the Deep Stack integration.
jwgizmo
Posts: 7
Joined: Wed Feb 24, 2021 12:12 am

Re: FPS Drops

Post by jwgizmo »

Have you seen this thread on ipcamtalk? :geek:

https://ipcamtalk.com/threads/common-is ... age.45933/
AMD R5, 16 GB, Win10, BI, 10 cams 3 Amcrest Turret 4K, 2 Annke CZ500 25X 4MP, 1 Annke C800 4K, 2 Amcrest indoor PTZ 1080P, 1 Dahua PTZ 1080P, 1 Amcrest 1080P Doorbell. 2 -1TB M.2 SSD, 1- 2TB HD|< 25 % load |1 27" 1080P, 1 28" 4K | Daily PC
Post Reply