Multiple Storage Drives

General discussion about Blue Iris
Post Reply
matgyver85
Posts: 2
Joined: Sun Feb 16, 2020 12:10 am

Multiple Storage Drives

Post by matgyver85 » Sun Feb 16, 2020 1:20 am

Hello, I am configuring BI5 to store video to four separate HDDs.

My questions pertains to the logic of the storage configuration.

If I...

Set each drive as a storage folder

Set a limit on size just under the size of drive

Apply an action to "move" to the next cascading folder/drive when size limit is reached;

Does this "move" the previoisly saved files from one drive to the next (which seems to be the worst thing to do if we're concerned about system resources or drive life), or

Does this "move" the storage location to the next drive (leaving previous drive in an idle state? (This seems it would be the best for system resources and drive longevity)

If this configuration would transfer the files from previous drive to next drive, then I would probably set it up to split my cameras up between the drives, so that I would have an even amount of cameras per drive.

I was thinking that this would probably be my best option, but this idea makes another setting seem irrational...

The guide for BI5 shows that NEW and Database folder locations should be on the fastest storage device due to "frequent access".

If I set groups of cams to save to each of my four drives;
Will this bypass the NEW folder and cause slow retrieval of new files which are no longer stored in the NEW folder on the fast drive?

Please advise to how the logic of the storage structure works.

Thanks for your time!
User avatar
TimG
Posts: 725
Joined: Tue Jun 18, 2019 10:45 am
Location: Nottinghamshire, UK.

Re: Multiple Storage Drives

Post by TimG » Fri Feb 21, 2020 6:34 pm

Hi, yes, if you set up all of the drives to move the files to the next drive when full, then it would indeed move the oldest files to the next drive in a cascade manner - not a good idea.

Many people only use one drive. In that case, you only need one folder, as there is no point in moving the files between folders (i.e New to Storage) on the same drive. It is your decision on how to set it up, but I presently have the BI database and all cameras recording to a separate SSD drive (the fast drive), and then move them to a slower storage drive as it gets full. Since my storage drive is a 4TB WD Purple, I could probably get away without the SSD. Quite simply, the SSD was in my spares box, so it went in to the PC !

How many cameras are you going to use ? That will determine whether you need to split the cameras between the drives or not.
Blue Iris v5.3.3.16 | Win10 x64 version 20H2 | Dahua IPC-HDW5231R-ZE, Foscam R2, Ertech 4MP, Neos2, 2 analogue cameras on Euresys Picolo Pro 2 | Intel i5-3330 CPU, 16GB Ram, Multiple SSD and HD | TVMosaic.
User avatar
Thixotropic
Posts: 539
Joined: Wed Sep 04, 2019 7:20 pm

Re: Multiple Storage Drives

Post by Thixotropic » Fri Feb 21, 2020 6:53 pm

matgyver85 wrote:
Sun Feb 16, 2020 1:20 am
If this configuration would transfer the files from previous drive to next drive, then I would probably set it up to split my cameras up between the drives, so that I would have an even amount of cameras per drive.
This is just my opinion, but I think in most cases using multiple drive arrays and moving data around is usually more trouble than it's worth. That's just my take on it.

I looked at doing cascade archiving, cloud uploads, etc and decided that for me it didn't make a lot of sense. I ended up just getting a small NS and dropping a couple of 4TB drives in it in a RAID configuration. It's well-protected but nothing super fancy.

My needs are fairly ordinary, so if you have lots of cameras or need bulletproof, no-fail storage with an accountability trail (like CoC stuff) then you may need to use something more elaborate.

Just my 2 cents, your mileage may vary, lol.
Blue Iris 5.x x64 | Windows 10 Pro x64 | 16GB RAM | i7-7700 3.6 GHz | 1TB HDD | 2TB RAID NAS | 9 Cameras | Almost Dual NIC | 2KVA UPS
matgyver85
Posts: 2
Joined: Sun Feb 16, 2020 12:10 am

Re: Multiple Storage Drives

Post by matgyver85 » Tue Oct 13, 2020 2:26 am

Thank you guys for the replies. Sorry, I didn't even know anyone had replied since I never received a notification. I literally stumbled back across my own thread while googling this issue again.

I was looking to use four drives instead of one for the simple reason that I had already purchased four 6TB drives for an old amcrest NVR. The nvr failed twice, so I decided to build my own NVR pc with Blue iris.
A 24TB HDD would be crazy expensive, if it even exists.
I have thought a bit about getting a NAS to save to, but again, $$.

Things have been working fairly well with BI except that my PC always seems to creep up to 100% CPU usage. Normally I sit around 30 to 60% cpu depending upon whether I am streaming decoded profiles or not.

I believe the cpu creep is related to the transferring of the stored clips from disc to disc to disc to disc.
I have switched to using only one drive, and so far the cpu creep is gone.

I may use windows to create one large virtual storage device out of the four drives which should eliminate the cpu creep, however, if one drive fails, all data is corrupted (per google).

I feel that the absolute best solution would be for BI to alter how the storage is handled.
Perhaps an option to "SWITCH" can be added to the "MOVE", "RECYCLE", or "DELETE" options under clips and archiving. The "SWITCH" option would switch from the first drive/folder to the next in the cascade to prevent excessive storage usage and cpu usage. After the last location in the cascade has been filled, the location would SWITCH to the first location overwriting the oldest files first.

BTW I have six 8mp cams and six 3mp cams. I prefer quality clips vs quantity. I get about 40 days of constant recording on most cams (all 8mp and a few 3mp) the rest only record motion events.

Hopefully one of these ideas pans out.

Thank you for the input!
IAmATeaf
Posts: 103
Joined: Mon Jun 17, 2019 7:48 pm

Re: Multiple Storage Drives

Post by IAmATeaf » Tue Oct 13, 2020 4:41 pm

Do what I do with 2 drives. Set each drive up in BI to use all of the space bar 5-10%, whatever you are happy with, don’t select the option to move but select the option to Delete which will let BI manage the space, also deselect the option for Clip limit age.

Do the same for the other drives. Remember that the Alerts folder is a special folder and cannot be renamed or used for clips storage. You can rename the folders by clicking on the folder name twice, again don’t rename Alerts.

Then within each cams config select the drives so that the cams are equally split to record against each drive.

Bit difficult to type and explain here but hopefully should make sense when in front of the BI console.

As others have pointed out there’s no point moving large files around in your BI PC as the task will just consume large amounts of CPU time and bog the disks down whilst the move is taking place.
matgyver85
Posts: 2
Joined: Sun Feb 16, 2020 12:10 am

Re: Multiple Storage Drives

Post by matgyver85 » Tue Oct 20, 2020 3:23 am

I mentioned the above option in my O.P.

, then I would probably set it up to split my cameras up between the drives, so that I would have an even amount of cameras per drive.

I was thinking that this would probably be my best option, but this idea makes another setting seem irrational...

The guide for BI5 shows that NEW and Database folder locations should be on the fastest storage device due to "frequent access".

If I set groups of cams to save to each of my four drives;
Will this bypass the NEW folder and cause slow retrieval of new files which are no longer stored in the NEW folder on the fast drive?


Doesn't this config bypass "NEW" folder on the "fast drive"?
Last edited by matgyver85 on Tue Oct 20, 2020 3:26 am, edited 1 time in total.
User avatar
TimG
Posts: 725
Joined: Tue Jun 18, 2019 10:45 am
Location: Nottinghamshire, UK.

Re: Multiple Storage Drives

Post by TimG » Tue Oct 20, 2020 4:09 pm

Hi,
Will this bypass the NEW folder and cause slow retrieval of new files which are no longer stored in the NEW folder on the fast drive?
Yes, it would bypass the "New" folder, and I think it must impact retrieval of new files, since they will now be on four separate slower drives. Does that matter ? Experimentation time, and let us know ! We are all learning here :D
Blue Iris v5.3.3.16 | Win10 x64 version 20H2 | Dahua IPC-HDW5231R-ZE, Foscam R2, Ertech 4MP, Neos2, 2 analogue cameras on Euresys Picolo Pro 2 | Intel i5-3330 CPU, 16GB Ram, Multiple SSD and HD | TVMosaic.
IAmATeaf
Posts: 103
Joined: Mon Jun 17, 2019 7:48 pm

Re: Multiple Storage Drives

Post by IAmATeaf » Tue Oct 20, 2020 9:14 pm

I have all my clips saved to a “slower” drive and haven’t really noticed any real impact on speed of access. I do tend to mainly use the mobile app, as do the rest of the family and nobody including my tech savvy sons have ever complained of clips access being slow or sluggish.
quest100
Posts: 3
Joined: Sat Oct 12, 2019 10:47 pm

Re: Multiple Storage Drives

Post by quest100 » Tue Oct 20, 2020 10:33 pm

My system is set up with 15 cameras constantly feeding three separate hard drives. I divided the cameras up such that each disk would start deleting video at about the same time - around 30 days. Two disks are external and connected via USB-C. Disks sizes are 10TB, 14TB, and 14TB. Each disk was set to reserve around 30 GB of free space. The reserve was picked to be greater than the sum of two video files from each camera on that disk. Nothing else is on these three drives.

My first iteration of this had the database and operating system on a (fourth) hard disk. Rebuilding the database for 10,000+ video files seemed to take forever so I moved them to a SSD. This has been running for 9 months with no problems.
Post Reply