Blue Iris Killing LAN Bandwidth

tobysnooker
Posts: 5
Joined: Sun Nov 14, 2021 11:05 pm

Blue Iris Killing LAN Bandwidth

Post by tobysnooker »

I have a fairly complicated network setup for a home, but Blue Iris is killing the bandwidth on the 2.5Gbps portion of the network (all wired, no WiFi)

BI5 is installed on a server running Windows Server 2022, which has a 10G ASUS NIC running in 2.5G mode.
This server is running as a NAS, VM host and is running Blue Iris.

I noticed that file transfers were getting really slow (relatively speaking). I shift around a lot of video projects to/from that server. With 2.5G I normally get about 280MB/second on file transfers. That's hitting a 1TB Optane cache drive, so I know the receiving end is not the problem. The sending end is also a 1TB Optane drive, so that's not the problem.

When Blue Iris is not running I get a constant 2.37Gbit/second using iperf3
When I spin up Blue Iris with all cameras disabled I get the same 2.37Gbit/second
When I start 1 camera the bandwidth drops to 2.0Gbit/second
When I add the second camera the bandwidth drops to 1.49Gbit/second
When I add the third camera the bandwidth drops to 722Mbit/second

File transfer speeds drop off a cliff and are much worse than those available bandwidth figures suggest.

Currently running 5.5.2.5, although the same behaviour is apparent with 5.5.1.20

The only change I made was going from a 2.5G NIC to a 10G NIC running in 2.5G mode.

But the only thing that kills the network is BlueIris. The bandwidth between other computers on the 2.5G network also degrades, just because Blue Iris is running with cameras added. I.e. Blue Iris is installed on computer A, running with three cameras active. The bandwidth between computers B and C degrades.

What is it that could be causing this problem?
IAmATeaf
Posts: 466
Joined: Mon Jun 17, 2019 7:48 pm

Re: Blue Iris Killing LAN Bandwidth

Post by IAmATeaf »

What cams and what resolution do you run? As that will dictate the network load per cam. Don’t forget that when the cams are enabled the cam will be constantly streaming regardless if you have 24/7 recording or rely upon motion detection.

TCP/IP handles collisions when 2 devices try to send data at the same time by backing off, I’m sure you already know this given what you have setup. So the cams are loading the network which will have an impact on any devices connected to the same network.

You can get around this by enabling VLANS and then tagging the cams on that VLAN but to do this you need suitable layer 2/3 switches or you could install a 2nd NIC and have all the cams connected via this 2nd NIC network.

I’ve tried to keep the above simple but I do both on my network even though I don’t strictly need the VLAN as the cams all connect via the 2nd NIC so are kept off my main home network.
Tommyt57
Posts: 15
Joined: Tue Sep 21, 2021 10:59 pm

Re: Blue Iris Killing LAN Bandwidth

Post by Tommyt57 »

I've had nothing but trouble with 5.5. Try downgrading to 5.4.9.18 and see if you have the same issues. Won't hurt and easy test. Also, not knowing your BI settings, it could be a configuration issue OR a camera issue, You could also disable motion detection on each camera and test your network.

Most new cameras have firmware that supports H265 that should lower the camera network usage. I've found that high bitrates can also flood the network.
IAmATeaf
Posts: 466
Joined: Mon Jun 17, 2019 7:48 pm

Re: Blue Iris Killing LAN Bandwidth

Post by IAmATeaf »

The bitrate will need to be set according to the cam resolution, no point in running it too low which will result in a lower quality recording.
tobysnooker
Posts: 5
Joined: Sun Nov 14, 2021 11:05 pm

Re: Blue Iris Killing LAN Bandwidth

Post by tobysnooker »

I've tried downgrading to 5.4.9.18 but had similar results. If anything the remaining bandwidth dropped by about 100Mbit/s.

The BI stats show cameras using 720kB/s, which is about 5.6Mbit/s, but I'm seeing bandwidth loss on the 2.5Gbit link of over 300 times that amount.
Windows Server Resource Monitor confirms BlueIris is only consuming 720kB/s so this really points to a network FUBAR that only manifests itself when Blue Iris is connected to cameras.

There are only two cameras connected, one hikvision and one a random Aliexpress board camera I built into the front door with a fisheye lens that looks like a peep-hole viewer thing. Both are H264 and can't do H265. I have a few other cameras but those aren't connected at present. The hikvision has all and every unwanted feature disabled (e.g. uPnP, SMTP, FTP saves, WiFi etc).
This all used to work fine with BI on a Win10 host using a 2.5Gb NIC and a 1Gb NIC before that. It's only when I upgraded to Server 2022 (thank you work MSDN) and the 10Gb NIC that things went wrong. I also swapped out my UDM for a UDM Pro and changed out the UniFi WAPs for Ruckus ones, but that shouldn't have an impact.

The cameras go to a managed PoE switch so I could create a VLAN for them on that, although it would be kind of useless as it then goes to an unmanaged switch that doesn't understand 802.1q. I've tried a couple of different network configs but still the same result. I tried disabling IPS and DPI on the internet gateway but that didn't work either.

The solution appears to be to run a new cable from the PoE switch used by the cameras to the BI server and plug it straight into another NIC (e.g. there is a spare 1Gb NIC in there). The cams and 1Gb NIC would be on a dedicated VLAN.
As a last resort I've tried changing NIC settings link interrupt moderation, buffer sizes etc. but still the same. Interestingly the available bandwidth goes right back up to 2.37Gbps after I've reset the NIC and then gradually slides back down to about 700Mbps over 20-30 seconds

It seems like a bit of unnecessary effort given that previously it simply worked for at least 4 years straight.
rfc805
Posts: 4
Joined: Sat Nov 13, 2021 6:19 pm

Re: Blue Iris Killing LAN Bandwidth

Post by rfc805 »

Are the cameras 1Gb eth, or 100Mb eth?

Understand that unless you have a pretty high end switch, the more 100mb ports talking the more bandwidth you're going to lose across the switch fabric due to rate conversion. Most consumer-grade and even SMB switches aren't going to have any sort of fancy buffering to deal with the rate conversion, so if the 100mb talkers are particularly active, you're going to drop throughput across the switch to a fraction of that 2.5gb.

If they're 1gb clients I doubt you'd even notice, but 100mbit clients you'll notice.

You could test this by streaming from the cameras with blue iris shut off, and see if you get similar results.
User avatar
TimG
Posts: 2098
Joined: Tue Jun 18, 2019 10:45 am
Location: Nottinghamshire, UK.

Re: Blue Iris Killing LAN Bandwidth

Post by TimG »

Is that yet another benefit of putting your cameras on a totally seperate LAN :idea:

TwoNicTim. Hmm, that's got a ring to it 8-)
Forum Moderator.
Problem ? Ask and we will try to assist, but please check the Help file.
tobysnooker
Posts: 5
Joined: Sun Nov 14, 2021 11:05 pm

Re: Blue Iris Killing LAN Bandwidth

Post by tobysnooker »

I've put another network cable from the switch that hosts the cameras (and other stuff), into a spare NIC in the BI server (in fact I tried two other NICs in it). Also set up traffic segmentation (D-Link switch), to isolate the two camera ports and the other port going to the server. Restarted everything, same problem and with a new one too: seems like the hikvision camera absolutely does not like being isolated from the internet and refuses to work. It's got a fixed IP but maybe it's stuck searching for the DNS/NTP/whatever server. Maybe because it can't see the gateway.

Turning off the firewall for private networks has helped somewhat in certain conditions.

So even with a theoretically dedicated network for the cameras I'm still seeing problems.

I might try VLANs next, but only one out of four switches are managed so not sure how they will handle the tags and I'll have to traverse two of them.
It could be a problem with Windows Server 2022 config somewhere
tobysnooker
Posts: 5
Joined: Sun Nov 14, 2021 11:05 pm

Re: Blue Iris Killing LAN Bandwidth

Post by tobysnooker »

A not-so-good news update on this:

I decided to turn off Blue Iris and install DW Spectrum, pulling feeds from three cameras. Remaining bandwidth is 1.87Gbps and I can copy files to the server at 220MB/s

So this points to some kind of issue with Blue Iris. Maybe a compatibility issue with Windows Server 2022?
Anyone have any thoughts on that?
rfc805
Posts: 4
Joined: Sat Nov 13, 2021 6:19 pm

Re: Blue Iris Killing LAN Bandwidth

Post by rfc805 »

Do you have a decent quality 10Gb NIC?

The reason I ask is you describe a result that is similar to software-driven NICs. Especially where turning off the SW firewall helps. That kind of hints that your NIC is CPU constrained, where a 10Gb NIC definitely needs hardware offload to function well. Without it, it only performs at low CPU load.

Maybe make sure you're using proper drivers and have the driver settings for hardware offloads enabled?

I'm not sure how Blue Iris could really influence throughput in that manner. The question is what CAN Blue Iris influence that could influence throughput. IE: Blue Iris using excessive CPU due to doing re-encoding which also depletes CPU for a software driven NIC.

Just guessing, but that's the viewpoint I'd work from to try to figure out what's going on.
Post Reply