Upgrading to a GTX 1080 Ti - any tips?

[rdxny]

I also have the disappearing GPU on the console bottom, but if I look at my GPU status in windows task manager, I get peaks in GPU usage when cameras get triggered in BI, so it looks like a display bug in BI only. (using .NET in CPAI with built in GPU)

Hmmm, I'm not seeing any activity in my GPU monitor, it's flat as a board even though it identifies the GPU as present. So, maybe it's not being used...?
-

gpu-2023-02-04_8-44-18.png

I would agree, it says zero usage on yours... Mine looks like this when busy (though I have integrated GPU, not nvidia, so that's a difference):

[PaulDaisy]

I do see spikes in GPU status on AI activity, but the CPAI server window appears to be a much more reliable indicator. Once I got CUDA to work reliably, the GPU processing is always displayed, along with 30-110 ms processing times (before CUDA they were at 3000-12000 ms):

I never see any GPU utilization when encoding video. I don't think BI uses GPU to do that.

[Thixotropic]

Well crap, I'm just not seeing it. Hmmm, not sure what to do at this point, lol.

I notice you have only Object Detection (YOLO) running.

I've only turned off Portrait Filter and still have Background Remover and Scene Classification running. Would you recommend turning those off for faster/better performance?

I kind of figured the Background Remover was needed, but maybe not?

-

And it is detecting stuff, but I expected lower times than this. That's why I'm not sure if BI is really using the GPU or not.
-

-

Finally though, there's the CPAI log. What's below is from a clean reboot. There's a line that says:

1:04:22 PM: Object Detection (YOLO): (General Exception) : The size of tensor a (32) must match the size of tensor b (52) at non-singleton dimension 3

I don't know if that's a serious error or not. (??) A few other items of interest in the log are in bold below:


API server is online
1:03:50 PM: Operating System: Microsoft Windows 10.0.19044
1:03:50 PM: Architecture: X64
1:03:50 PM: Environment:
1:03:50 PM: Platform: windows
1:03:50 PM: In Docker: False
1:03:50 PM: App DataDir: C:\ProgramData\CodeProject\AI
1:03:57 PM: Video adapter info:
1:03:57 PM: Name - Intel(R) HD Graphics 630
1:03:57 PM: Device ID - VideoController1
1:03:57 PM: Adapter RAM - 1,024 MB
1:03:57 PM: Adapter DAC Type - Internal
1:03:57 PM: Driver Version - 31.0.101.2111
1:03:57 PM: Video Processor - Intel(R) HD Graphics Family
1:03:57 PM: Video Architecture - VGA
1:03:57 PM: Video Memory Type - Unknown
1:03:57 PM: GPU 3D Usage - 1%
1:03:57 PM: GPU RAM Usage - 68.63 MB
1:03:57 PM: Name - NVIDIA GeForce GTX 1080 Ti
1:03:57 PM: Device ID - VideoController2
1:03:57 PM: Adapter RAM - 4 GB
1:03:57 PM: Adapter DAC Type - Integrated RAMDAC
1:03:57 PM: Driver Version - 31.0.15.1601
1:03:57 PM: Video Processor - NVIDIA GeForce GTX 1080 Ti
1:03:57 PM: Video Architecture - VGA
1:03:57 PM: Video Memory Type - Unknown
1:03:57 PM: GPU 3D Usage - 4%
1:03:57 PM: GPU RAM Usage - 68.57 MB
1:03:57 PM: BackendProcessRunner Start
1:04:00 PM: Attempting to start Scene Classification
1:04:00 PM:
1:04:00 PM: Module 'Scene Classification' (ID: SceneClassification)
1:04:00 PM: Active: True
1:04:00 PM: GPU: Support enabled
1:04:00 PM: Parallelism: 1
1:04:00 PM: Platforms: windows,linux,macos,macos-arm,docker
1:04:00 PM: Runtime: python37
1:04:00 PM: Queue: scene_queue
1:04:00 PM: Start pause: 1 sec
1:04:00 PM: Valid: True
1:04:00 PM: Environment Variables
1:04:00 PM: APPDIR = %MODULES_PATH%\Vision\intelligencelayer
1:04:00 PM: CPAI_MODULE_SUPPORT_GPU = True
1:04:00 PM: DATA_DIR = %DATA_DIR%
1:04:00 PM: MODE = MEDIUM
1:04:00 PM: MODELS_DIR = %MODULES_PATH%\Vision\assets
1:04:00 PM: PROFILE = desktop_gpu
1:04:00 PM: TEMP_PATH = %MODULES_PATH%\Vision\tempstore
1:04:00 PM: USE_CUDA = True
1:04:00 PM: VISION-SCENE = True
1:04:00 PM: YOLOv5_VERBOSE = false
1:04:00 PM:
1:04:00 PM: Started Scene Classification backend
1:04:01 PM: Latest version available is 2.0.7-Beta
1:04:01 PM: Attempting to start Face Processing
1:04:01 PM: Attempting to start Background Remover
1:04:01 PM:
1:04:01 PM: Module 'Background Remover' (ID: BackgroundRemover)
1:04:01 PM: Active: True
1:04:01 PM: GPU: Support disabled
1:04:01 PM: Parallelism: 1
1:04:01 PM: Platforms: windows,linux,docker,macos,macos-arm
1:04:01 PM: Runtime: python39
1:04:01 PM: Queue: removebackground_queue
1:04:01 PM: Start pause: 0 sec
1:04:01 PM: Valid: True
1:04:01 PM: Environment Variables
1:04:01 PM: U2NET_HOME = %MODULES_PATH%/BackgroundRemover/models
1:04:01 PM:
1:04:01 PM: Started Background Remover backend
1:04:01 PM: Attempting to start Object Detection (YOLO)
1:04:01 PM:
1:04:01 PM: Module 'Object Detection (YOLO)' (ID: ObjectDetectionYolo)
1:04:01 PM: Active: True
1:04:01 PM: GPU: Support enabled
1:04:01 PM: Parallelism: 0
1:04:01 PM: Platforms: all
1:04:01 PM: Runtime: python37
1:04:01 PM: Queue: detection_queue
1:04:01 PM: Start pause: 1 sec
1:04:01 PM: Valid: True
1:04:01 PM: Environment Variables
1:04:01 PM: APPDIR = %MODULES_PATH%\ObjectDetectionYolo
1:04:01 PM: CPAI_CUDA_DEVICE_NUM = 0
1:04:01 PM: CPAI_HALF_PRECISION = Enable
1:04:01 PM: CPAI_MODULE_SUPPORT_GPU = True
1:04:01 PM: CUSTOM_MODELS_DIR = %MODULES_PATH%\ObjectDetectionYolo\custom-models
1:04:01 PM: MODELS_DIR = %MODULES_PATH%\ObjectDetectionYolo\assets
1:04:01 PM: MODEL_SIZE = Medium
1:04:01 PM: USE_CUDA = True
1:04:01 PM: YOLOv5_VERBOSE = false
1:04:01 PM:
1:04:01 PM: Started Object Detection (YOLO) backend
1:04:02 PM: Attempting to start Object Detection (.NET)
1:04:02 PM: Attempting to start Portrait Filter
1:04:02 PM: Latest version available is 2.0.7-Beta
1:04:02 PM: *** A new version 2.0.7-Beta is available **
1:04:09 PM: Background Remover: Background Remover started.
1:04:11 PM: scene.py: Vision AI services setup: Retrieving environment variables...
1:04:11 PM: scene.py: GPU in use: NVIDIA GeForce GTX 1080 Ti
1:04:11 PM: Scene Classification: Scene Classification started.
1:04:16 PM: detect_adapter.py: APPDIR: C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo
1:04:16 PM: detect_adapter.py: CPAI_PORT: 32168
1:04:16 PM: detect_adapter.py: MODEL_SIZE: medium
1:04:16 PM: detect_adapter.py: MODELS_DIR: C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\assets
1:04:16 PM: detect_adapter.py: support_GPU: True
1:04:16 PM: detect_adapter.py: use_CUDA: True
1:04:16 PM: Object Detection (YOLO): Running init for Object Detection (YOLO)
1:04:16 PM: Object Detection (YOLO): Object Detection (YOLO) started.
1:04:22 PM: Object Detection (YOLO): Detecting using ipcam-combined
1:04:22 PM: Object Detection (YOLO): Detecting using ipcam-combined
1:04:22 PM: Object Detection (YOLO): Detecting using ipcam-combined
1:04:22 PM: Object Detection (YOLO): Detecting using ipcam-combined
1:04:22 PM: Object Detection (YOLO): Detecting using ipcam-combined
1:04:22 PM: Object Detection (YOLO): Queue and Processing Object Detection (YOLO) command 'custom' took 5546ms
1:04:22 PM: Object Detection (YOLO): Queue and Processing Object Detection (YOLO) command 'custom' took 5544ms
1:04:22 PM: Object Detection (YOLO): Queue and Processing Object Detection (YOLO) command 'custom' took 5549ms
1:04:22 PM: Object Detection (YOLO): Queue and Processing Object Detection (YOLO) command 'custom' took 5544ms
1:04:22 PM: Object Detection (YOLO): (General Exception) : The size of tensor a (32) must match the size of tensor b (52) at non-singleton dimension 3
1:04:22 PM: Object Detection (YOLO): Queue and Processing Object Detection (YOLO) command 'custom' took 6077ms
1:04:47 PM: Latest version available is 2.0.7-Beta

[PaulDaisy]

I don't know what to make of the tensor size error. That may be a question for the CPAI board. But your processing times in the BI log are definitely on the order of what I see with a GPU, and your CPAI console is saying that it is using the GPU, so I think it is working as it should. My CPU times were on the order of seconds, these are 70-120 ms.
YOLO is the only one I have enabled and it is working just fine without the background enabled. I also removed all custom models except combined (and delivery, which doesn't work very well).

[Thixotropic]

It turns out the reason that Windows won't display the GPU usage in Task Manager is because the driver's WDDM level is too low. It's a known problem, so at least I can stop worrying about that.

https://www.addictivetips.com/windows-t ... indows-10/

"Your chip’s driver must support WDDM version 2.0 or above. If your GPU’s driver doesn’t support WDDM version 2.0 or above, you simply cannot view your GPU performance in the task manager.

If there is no GPU in the Task Manager in Windows 10, try the following.

Open the run dialog box via the Win+R keyboard shortcut. Type in Dxdiag.exe and tap the Enter key. If you see a prompt of any sort on you screen, click ‘Yes’ on it.

When the dxdiag tool opens, go to the Display tab. In the Drivers section, look for Driver Model. If it doesn’t say WDDM 2.0, or anything above 2.0, then your driver is not compatible with this new feature."

(Not my system, this is from the article)

[Thixotropic]

Okay, after a little more poking around and whatnot, I'm seeing daylight detection times more along the lines of what I was expecting- 60ms 70ms detection times and a rough average time of ~90ms or so.

Things may be slower than the normal lowest possible times in part because I mostly use main streams and only a couple substreams. It may be that because CPAI is forced to use the higher res feed, that might slowing things down a little (maybe). But overall I'm pretty satisfied with these times.

With that said, my current hardware might change for the better shortly.

My son is getting a new motherboard, CPU, and graphics card, and he's willing to give me his old CPU super cheap (the well-known Daddy Discount). His old CPU is several generations above what I have in my personal PC right now.

So the idea is to take his old CPU and put it into my personal PC, then take the old CPU from my personal PC and put that in the Blue Iris box. That will also be several steps up in CPU for the BI box as well.

Finally, he's offered me his "old" 2080 TI card, but I'm not sure just how much improvement I'd see in Blue Iris and CPAI over the 1080 TI. A lot? A little? What kind of difference between the cards would you expect to see? 10 or 20% maybe?

[TimG]

26% according to this website.

TDP is 250W for both cards, so no issues there.

[noloboy]

@thixotropic

I'm kind of the same boat. I have a 12700k + Quadro P2000 and I'm trying to switch over to CPAI but like your experience it keeps on crashing as soon as I click on "STOP NOW" then "OK" from the settings for AI tab. Even today the 5.7 version just got released and its the same issue.

I will be downgrading both BI 5.6.8.4 and CPAI to 1.6.8 but I can't seem to find the link to download CPAI 1.6.8. Do you mind sharing the link please?

[Thixotropic]

I will be downgrading both BI 5.6.8.4 and CPAI to 1.6.8 but I can't seem to find the link to download CPAI 1.6.8. Do you mind sharing the link please?

I can't find the link to 1.68, but I have a saved copy that I've put up on a file share.

Also, 2.06 and 2.07 are supposed to be viable with BI 5.6.8.4 but I haven't tried either of them yet.

Here's the WeTransfer link to the files below (~1.7MB total): https://we.tl/t-zA1tHzNzyV

CodeProject.AI.Server-1.6.8.0.zip
CodeProject.AI.Server-2.0.6.exe
CodeProject.AI.Server-2.0.7.exe

I have not tried either 2.06 or 2.07, but the chatter on the CPAI board seems to indicate that they'll work with BI 5.6.8.4. (They actually mention 2.02 but I've not been able to find that version.)

[Thixotropic]

it keeps on crashing as soon as I click on "STOP NOW" then "OK" from the settings for AI tab.

Yes, that never ended well for me either. All sorts of weird stuff would start happening and then it was reboot time. It never ever recovered from that, not even once. Same deal with stopping it from the CPAI web interface, it was always fatal in the end.