100% Memory usage while converting

[MadsRavnJensen]

Hi,

I have a huge problem with 100% memory usage while converting to MP4. I use direct to disc recording, and do not re-encode the video. I only convert videos that are not actively recording e.g. yesterdays recording.
This has worked nicely before, but now I see BlueIris 5 eating up all the memory while converting, and not releasing the memory afterwards.
This causes all kinds of issues, sometimes I am unable to convert the recordings, and I need to restart BlueIris to be able to convert again.

I do convert on a daily basis, and the recordings are 24 hours continuous recording. I you are wondering why I record so much I can inform that it is for a science project.

Does anyone else experience this behaviour ?

[Thixotropic]

I use a powershell script and ffmpeg to do my conversions to MP4 but yeah, the CPU spikes when the conversion is underway. Not 100%, but it definitely goes up a fair amount. I'll post the powershell script here if anyone wants to have a look at it.

[MikeBwca]

In your 'Convert/Export' window, click the 'Encoder Profile' 'Configure' button.
There are so many options that will effect the conversion, and, cpu usage.
Experiment here. But first take a screenshot so you can set the settings back.
I'd suggest leaving the default profile alone, and changing one of the others.

You can set up to 4 export encoding profiles, and, export using different profiles to compare performance, size...

[MadsRavnJensen]

Thank you for the inputs,

It is the memory usage that gives me an headache.
Especially because the memory insn't released after a conversion so sometimes/often I have to close down BlueIris to force the memory to be released.
And then I can convert a recording again.

If I look at the CPU usage during export/convert hardly anything is going on ~25% cpu load total on a i7-8700 at 3.2GHz. with 16GB memory. Where BlueIris consumes around 13,5GB!

For me it looks like a memory leak, that will very slowly be cleaned up, not entirely but the memory usage will go down over time (Hours).

Regarding settings I use the default settings, I haven't even touched them

I would appreciate to see the ffmpeg powershell script as I am considering to do the conversion of the files on a seperate computer. I was not aware that it was possible to convert the bvr file directly using ffmpeg.

I am at the moment completly unable to export from within BlueIris, even after a reset and database cleanup. I have used BlueIris for month without this problem. I really don't understand what is wrong.

[Thixotropic]

I would appreciate to see the ffmpeg powershell script as I am considering to do the conversion of the files on a seperate computer. I was not aware that it was possible to convert the bvr file directly using ffmpeg.

Unfortunately the forum doesn't allow attaching anything but image files, so here you go.
Save this code as "bvr_converter.ps1" and then set the config parameters at the top of the script. A short README file is below the code.

Code: Select all

<#
bvr_converter.ps1 ver. 1.10 | 3/2020 by Mike, Thixotropic, & LDM

This powershell script converts .BVR files to .MP4 files, preserving the original file's timestamp. 
Options include keeping the original size or scaling to 1/2, 1/3, or 1/4 of the original 
video size/resolution. 

You can manually convert a single file with a command line like this: 

  ffmpeg -n -loglevel error -i inputfile.bvr -vcodec libx264 -crf 30 outputfile.mp4

For more info on using ffmpeg, see: https://www.ffmpeg.org/
For more on the CRF parameter, see: https://slhck.info/video/2017/02/24/crf-guide.html

#>

################################
# Configuration settings start
################################

#######
# Source directory where the .BVR files to be converted are found. DO NOT use a trailing slash (\).
# For example, "C:\BlueIris\Stored"
$source_dir = "C:\BlueIris\Stored"

####### 
# Target directory where the converted .MP4 files will stored. DO NOT use a trailing slash (\).
# You will probably want to change this to another directory, a network drive, attached storage, etc. 
# For example, "C:\BlueIris\Stored"
$target_dir = "C:\BlueIris\Stored"


#######
# Pause time between conversions. Useful to avoid overloading the PC doing the converting.
# Value is in seconds; typically 1 or 2 seconds is plenty, but adjust as needed. 
$pause_time = 2


####### 
# The actual command line params for conversion. Don't change these if you don't know what you're doing. 
# UNCOMMENT ONE AND ONLY ONE LINE BELOW

# convert video with no size scaling - this is the default. Files are normally smaller by 20 to 50 percent.
$ffmpeg_params = "-n -loglevel error -i $source_dir\$oldvid -vcodec libx264 -crf 30 $source_dir\$newvid"

# Convert and scale video to 1/2 size - produces MUCH smaller videos with a minor loss of resolution.
#$ffmpeg_params = "-n -loglevel error -i $source_dir\$oldvid -vcodec libx264 -crf 30 -vf \"scale=iw/2:ih/2\" $source_dir\$newvid"

# Convert and also scale video to 1/3 size - produces much, much smaller videos with some loss of resolution.
#$ffmpeg_params = "-n -loglevel error -i $source_dir\$oldvid -vcodec libx264 -crf 30 -vf \"scale=iw/3:ih/3\" $source_dir\$newvid"

# Convert and also scale video to 1/4 size - produces ridiculously small videos suitable for sending by text ot email. 
#$ffmpeg_params = "-n -loglevel error -i $source_dir\$oldvid -vcodec libx264 -crf 30 -vf \"-vf "scale=iw/4:ih/4\" $source_dir\$newvid"

################################
# Configuration settings stop
################################


# pop into the source directory
cd $source_dir 

echo "Source directory: $source_dir "
echo "Target directory: $target_dir "

# get a list of all the .BVR files
$oldvids = Get-ChildItem -Path $source_dir -Filter "*.bvr"

# step through the .BVR files and convert them
foreach ($oldvid in $oldvids) {

    # get the timestamp of the original file 
    $orgtimestamp = (Get-Item $oldvid).LastWriteTime.toString()

    # change the extension from .bvr to .mp4 
    $newvid = [io.path]::ChangeExtension($oldvid, '.mp4')
    
    # print the source file name and timestamp for reference. You can comment this out if you want. 
    echo "Video source file: $source_dir\$oldvid "
    echo "Timestamp: $orgtimestamp "

    echo "FFMPEG comand: ffmpeg $ffmpeg_params"

    # this is where all the magic happens...
    Invoke-Expression "ffmpeg $ffmpeg_params"

    # write the original timestamp to the converted file 
    (Get-Item $newvid).LastWriteTime=($orgtimestamp)

    # pause time, if any
    if($pause_time -gt 0){Start-Sleep -s $pause_time}
    
    # move the file? 
    if($source_dir -ne $target_dir){
    # move the file to the destination target directory 
        echo "Moving file $newvid..."
        Move-Item -Path $source_dir\$newvid -Destination $target_dir\$newvid
    }


}

echo "Conversion complete."

Here's a short readme file:

Code: Select all

BVR Converter converts Blue Iris .BVR files to .MP4 fideo files while preserving their original timestamp. It also typically reduces their size, in some cases dramatically. Converting to the MP4 format allows the files to be provided to others to view without having to install Blue Iris. It can process an entire directory, file by file, and optionally move the files to another directory for storage (such as a NAS, an external hard drive, thumb drive, etc). 

This powershell script converts .BVR files to .MP4 files, preserving the original file's timestamp. 
Options include keeping the original size or scaling to 1/2, 1/3, or 1/4 of the original video size/resolution.  

The only external component required is the free utility "ffmpeg", which may be found here: 

    https://www.ffmpeg.org/

and downloaded from here: 

    https://www.ffmpeg.org/download.html

It's recommended that you get the pre-compiled binary instead of the source code. Versions of ffmpeg are available for Windows, Linux, and Mac OS. The powershell script is intended to be run under Windows 8 or greater. 

Install ffmpeg in a location that's in your computer's PATH variable so it can be run from anywhere, or install it somewhere else and add its location to your PATH variable. If you opt not to do either of these then you'll need to modify the bvr_converter.ps1 file so the full path to ffmpeg is referenced in the script. 

-------------
Configuration
-------------

The "source" directory is where the .BVR files to be converted are found. DO NOT use a trailing slash (\).
For example: "C:\BlueIris\Stored" 

The "target" directory is where the converted .MP4 files will stored. DO NOT use a trailing slash (\).
For example: "C:\backup_video" 
You can set this to a different directory, a NAS, attached storage, or other remote location if you want. 

The "pause" time is an optional time to wait between each file conversion. This may be useful to avoid overloading the PC that's doing the converting. The "pause" time value is set in seconds, and typically 1 or 2 seconds is plenty, but adjust as needed. 
For example: $pause_time = 2 

Command line settings for ffmpeg
There are several sample command line parameters for conversion. Don't change these if you don't know what you're doing. 
Uncomment ONE and only ONE of the parameter lines. If you uncomment more than one, the last uncommented one will be used. 
For the most reduction in file size, use one of the settings that scales the image. 
--------------

[MadsRavnJensen]

Thank you for the script, it will be usefull

I appreciate your help.

Hae a nice easter, and stay healthy

[MadsRavnJensen]

GREAT NEWS! For me at least

Update 5.2.5.0 x64 10/4/20 has fixed the issue. Memory stays at approximately 25% usage and it even seems that BlueIris uses a little less memory.

Huge thanks to BlueIris for fixing this.

[mlisted]

@Thixotropic

Do you know how to make ffmpeg do what Blue Iris does, that is, output an MP4 without re-encoding? I'm doing direct-to-disc anyway, and Blue Iris exports without re-encoding, so the MP4 simply has be embedded in the BVR in its original encoding.

[mlisted]

Also seems the audio isn't included in the ffmpeg output. Is that true for you too?