Upgrading to a GTX 1080 Ti - any tips?

General discussion about Blue Iris

Moderators: HeneryH, TimG

User avatar
Thixotropic
Posts: 891
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by Thixotropic » Thu Feb 02, 2023 2:48 pm

PaulDaisy wrote: Thu Feb 02, 2023 2:11 am "Detecting using ipcam-combined, processing time -20 ms" :D
Heh heh, I can hope. :)

At this point, CPAI doesn't appear to be using *any* models, so I'm not seeing any detection events. This is what I think the problem is...

After installing everything and rebooting the system, everything appears to be working, but it's not. Take a look at this image:
-
GPU-AI-settings-2023-02-01_17-57-24.png
The "custom models" path is blank. Clicking on the 3-dot button, I get a popup that says no models are currently loaded, and to restart the AI to see what models are being used.

Edit: Even just clicking that 3-dot button puts the system into a weird, unstable, semi-frozen mode that it can't recover from. This is before even restarting the AI. Clicking that button is the Click Of Death.

When I restart the AI, everything goes to crap. The system freezes, unfreezes, and then nothing works properly. Everything is laggy and frozen. The only cure is to reboot the PC. That used to result in the "custom models" path populating and showing "ipcam-combined" in it, but it doesn't do that anymore, it's always blank.

The last couple of versions of BI had this same behavior, after a lot of crunchiness and reboots, the custom model would (finally) appear in the path and things would start working.

The only change I made to CPAI after installation was to move all the models I didn't want (ipcam-general.pt, license-plate.pt, etc) to a temp directory. This is the same change I made with the last version before installing the 1080ti card, and that seemed to work fine.

So I dunno...I have the GPU installed and working, CUDA is installed, etc etc, but I can't seem to get it to find the custom model that it shipped with. (??) It's in

C:\Program Files\CodeProject\CPAI\Analysis Layer\ObjectDetectionYolo\custom-models\ipcam-combined.pt

Same location as before but BI can't/won't load it.

So....what to do from here? Move the unused models back into the custom-models directory? I can't see how that would make anything better, but...?
Blue Iris 5.x | Windows 10 Pro Slim | 16GB RAM | i7-7700 3.6 GHz | GTX 1080Ti FE | 8TB RAID NAS | 9 Cams | 2KVA UPS
PaulDaisy
Posts: 82
Joined: Mon Jan 16, 2023 5:06 pm

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by PaulDaisy » Fri Feb 03, 2023 12:21 am

Thixotropic wrote: Thu Feb 02, 2023 2:48 pm The "custom models" path is blank. Clicking on the 3-dot button, I get a popup that says no models are currently loaded, and to restart the AI to see what models are being used.

Edit: Even just clicking that 3-dot button puts the system into a weird, unstable, semi-frozen mode that it can't recover from. This is before even restarting the AI. Clicking that button is the Click Of Death.

When I restart the AI, everything goes to crap. The system freezes, unfreezes, and then nothing works properly. Everything is laggy and frozen. The only cure is to reboot the PC. That used to result in the "custom models" path populating and showing "ipcam-combined" in it, but it doesn't do that anymore, it's always blank.
Same exact thing happened to me with the 5.6.9.x. I downgraded to the "last stable" 5.6.8.4 and everything went back to working as it used to.
User avatar
Thixotropic
Posts: 891
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by Thixotropic » Fri Feb 03, 2023 2:33 am

PaulDaisy wrote: Fri Feb 03, 2023 12:21 am Same exact thing happened to me with the 5.6.9.x. I downgraded to the "last stable" 5.6.8.4 and everything went back to working as it used to.
Thank you for this. After reading that, I rolled back to 5.6.8.4, and yes, everything seems to have started working again.

I notice on the AI tab, 5.6.8.4 has a checkbox labeled "CUDA", on 5.6.9.x it's labeled "GPU". Maybe the latest version is expanding to other services similar to CUDA.

The lower stats bar now shows the GPU being used and the CPU is way back down to 25~30%, but keep reading. All is not well. :(

Also, the custom path field now correctly shows the path to the custom model directory, so the next step is to move the unneeded models out of the directory, reboot, and see if everything still works as expected.

EDIT: I may have spoken too soon...

After the system sits running for about 5 minutes, CPAI appears to stop using the GPU.

The GPU indicator disappears from the status bar and never comes back. This after a clean reboot and not touching the system or interacting with it in any way. (??)

If I open the BI Status log, I see errors like "AI not responding" and nothing is being detected.

The log is indicating something in a particular file, shown in bold in the code below (the full window is below that).

07:36:37:Started Object Detection (YOLOv5 6.2) module
07:36:37:detect_adapter.py: Traceback (most recent call last):
07:36:37:detect_adapter.py: File "C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\detect_adapter.py", line 16, in
07:36:37:detect_adapter.py: from analysis.codeprojectai import CodeProjectAIRunner
07:36:37:detect_adapter.py: File "../../SDK/Python\analysis\codeprojectai.py", line 36, in
07:36:37:detect_adapter.py: from analysis.requestdata import AIRequestData
07:36:37:detect_adapter.py: File "../../SDK/Python\analysis\requestdata.py", line 8, in
07:36:37:detect_adapter.py: from PIL import Image
07:36:37:detect_adapter.py: ImportError: cannot import name 'Image' from 'PIL' (unknown location)
07:36:37:Module ObjectDetectionYolo has shutdown
07:36:37:detect_adapter.py: has exited

07:36:39:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...tectionYolo\detect_adapter.py"
07:36:39:
07:36:39:Module 'Object Detection (YOLOv5 6.2)' (ID: ObjectDetectionYolo)

07:36:39:Active: True
07:36:39:GPU: Support enabled
07:36:39:Parallelism: 0
07:36:39:Platforms: all
07:36:39:FilePath: detect_adapter.py
07:36:39:ModulePath: ObjectDetectionYolo
07:36:39:Install: PreInstalled
07:36:39:Runtime: python37
07:36:39:Queue: objectdetection_queue
07:36:39:Start pause: 1 sec


errors-2023-02-02_18-42-05.png

Code: Select all

07:36:35:Operating System: Windows (Microsoft Windows 10.0.19044)
07:36:35:CPUs:             1 CPU x 4 cores. 8 logical processors (x64)
07:36:35:GPU:              NVIDIA GeForce GTX 1080 Ti (11 GiB) (NVidia)
07:36:35:                  Driver: 516.01 CUDA: 11.7 Compute: 6.1
07:36:35:System RAM:       16 GiB
07:36:35:Target:           Windows
07:36:35:BuildConfig:      Release
07:36:35:Execution Env:    Native
07:36:35:Runtime Env:      Production
07:36:35:.NET framework:   .NET 7.0.2
07:36:35:App DataDir:      C:\ProgramData\CodeProject\AI
07:36:36:System GPU info:
07:36:36:  GPU 3D Usage       0%
07:36:36:  GPU RAM Usage      192 MiB
07:36:36:Video adapter info:
07:36:36:  Intel(R) HD Graphics 630:
07:36:36:    Adapter RAM        1,024 MiB
07:36:36:    Driver Version     31.0.101.2111
07:36:36:    Video Processor    Intel(R) HD Graphics Family
07:36:36:  NVIDIA GeForce GTX 1080 Ti:
07:36:36:    Adapter RAM        4 GiB
07:36:36:    Driver Version     31.0.15.1601
07:36:36:    Video Processor    NVIDIA GeForce GTX 1080 Ti
07:36:36:------------------------------------------------------------------
07:36:36:ROOT_PATH               = C:\Program Files\CodeProject\AI
07:36:36:MODULES_PATH            = C:\Program Files\CodeProject\AI\AnalysisLayer
07:36:36:DOWNLOADED_MODULES_PATH = C:\Program Files\CodeProject\AI\modules
07:36:36:PYTHON_BASEPATH         = C:\Program Files\CodeProject\AI\AnalysisLayer\bin\windows\%PYTHON_RUNTIME%\venv
07:36:36:PYTHON_PATH             = C:\Program Files\CodeProject\AI\AnalysisLayer\bin\windows\%PYTHON_RUNTIME%\venv\scripts\Python
07:36:36:Temp Dir:                C:\WINDOWS\TEMP\
07:36:36:Data Dir:                C:\ProgramData\CodeProject\AI
07:36:36:------------------------------------------------------------------
07:36:36:ModuleRunner Start
07:36:36:Starting Background AI Modules
07:36:37:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...tectionYolo\detect_adapter.py"
07:36:37:
07:36:37:Module 'Object Detection (YOLOv5 6.2)' (ID: ObjectDetectionYolo)
07:36:37:Active:      True
07:36:37:GPU:         Support enabled
07:36:37:Parallelism: 0
07:36:37:Platforms:   all
07:36:37:FilePath:    detect_adapter.py
07:36:37:ModulePath:  ObjectDetectionYolo
07:36:37:Install:     PreInstalled
07:36:37:Runtime:     python37
07:36:37:Queue:       objectdetection_queue
07:36:37:Start pause: 1 sec
07:36:37:Valid:       True
07:36:37:Environment Variables
07:36:37:APPDIR                  = %CURRENT_MODULE_PATH%
07:36:37:CPAI_CUDA_DEVICE_NUM    = 0
07:36:37:CPAI_HALF_PRECISION     = Enable
07:36:37:CPAI_MODULE_SUPPORT_GPU = True
07:36:37:CUSTOM_MODELS_DIR       = %CURRENT_MODULE_PATH%/custom-models
07:36:37:MODELS_DIR              = %CURRENT_MODULE_PATH%/assets
07:36:37:MODEL_SIZE              = Medium
07:36:37:USE_CUDA                = True
07:36:37:YOLOv5_AUTOINSTALL      = false
07:36:37:YOLOv5_VERBOSE          = false
07:36:37:
[b]07:36:37:Started Object Detection (YOLOv5 6.2) module
07:36:37:detect_adapter.py: Traceback (most recent call last):
07:36:37:detect_adapter.py:   File "C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\detect_adapter.py", line 16, in 
07:36:37:detect_adapter.py:     from analysis.codeprojectai import CodeProjectAIRunner
07:36:37:detect_adapter.py:   File "../../SDK/Python\analysis\codeprojectai.py", line 36, in 
07:36:37:detect_adapter.py:     from analysis.requestdata import AIRequestData
07:36:37:detect_adapter.py:   File "../../SDK/Python\analysis\requestdata.py", line 8, in 
07:36:37:detect_adapter.py:     from PIL import Image
07:36:37:detect_adapter.py: ImportError: cannot import name 'Image' from 'PIL' (unknown location)
07:36:37:Module ObjectDetectionYolo has shutdown
07:36:37:detect_adapter.py: has exited
07:36:39:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...tectionYolo\detect_adapter.py"
07:36:39:
07:36:39:Module 'Object Detection (YOLOv5 6.2)' (ID: ObjectDetectionYolo)[/b]
07:36:39:Active:      True
07:36:39:GPU:         Support enabled
07:36:39:Parallelism: 0
07:36:39:Platforms:   all
07:36:39:FilePath:    detect_adapter.py
07:36:39:ModulePath:  ObjectDetectionYolo
07:36:39:Install:     PreInstalled
07:36:39:Runtime:     python37
07:36:39:Queue:       objectdetection_queue
07:36:39:Start pause: 1 sec
07:36:39:Valid:       True
07:36:39:Environment Variables
07:36:39:APPDIR                  = %CURRENT_MODULE_PATH%
07:36:39:CPAI_CUDA_DEVICE_NUM    = 0
07:36:39:CPAI_HALF_PRECISION     = Enable
07:36:39:CPAI_MODULE_SUPPORT_GPU = True
07:36:39:CUSTOM_MODELS_DIR       = %CURRENT_MODULE_PATH%/custom-models
07:36:39:MODELS_DIR              = %CURRENT_MODULE_PATH%/assets
07:36:39:MODEL_SIZE              = Medium
07:36:39:USE_CUDA                = True
07:36:39:YOLOv5_AUTOINSTALL      = false
07:36:39:YOLOv5_VERBOSE          = false
07:36:39:
07:36:39:Started Object Detection (YOLOv5 6.2) module
07:36:39:detect_adapter.py: Traceback (most recent call last):
07:36:39:detect_adapter.py:   File "C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\detect_adapter.py", line 16, in 
07:36:39:detect_adapter.py:     from analysis.codeprojectai import CodeProjectAIRunner
07:36:39:detect_adapter.py:   File "../../SDK/Python\analysis\codeprojectai.py", line 36, in 
07:36:39:detect_adapter.py:     from analysis.requestdata import AIRequestData
07:36:39:detect_adapter.py:   File "../../SDK/Python\analysis\requestdata.py", line 8, in 
07:36:39:detect_adapter.py:     from PIL import Image
07:36:39:detect_adapter.py: ImportError: cannot import name 'Image' from 'PIL' (unknown location)
07:36:39:Module ObjectDetectionYolo has shutdown
07:36:39:detect_adapter.py: has exited
07:36:40:Installing Initial Modules.
07:36:40:Client request 'list-custom' in the queue (...aeaf2e)
07:36:41:Latest version available is 2.0.7-Beta
07:36:41:Current Version is 2.0.7-Beta
07:36:41:Server: This is the latest version
07:36:57:Latest version available is 2.0.7-Beta
18:10:19:Latest version available is 2.0.7-Beta
18:11:32:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...tectionYolo\detect_adapter.py"
18:11:32:
18:11:32:Module 'Object Detection (YOLOv5 6.2)' (ID: ObjectDetectionYolo)
18:11:32:Active:      True
18:11:32:GPU:         Support enabled
18:11:32:Parallelism: 0
18:11:32:Platforms:   all
18:11:32:FilePath:    detect_adapter.py
18:11:32:ModulePath:  ObjectDetectionYolo
18:11:32:Install:     PreInstalled
18:11:32:Runtime:     python37
18:11:32:Queue:       objectdetection_queue
18:11:32:Start pause: 1 sec
18:11:32:Valid:       True
18:11:32:Environment Variables
18:11:32:APPDIR                  = %CURRENT_MODULE_PATH%
18:11:32:CPAI_CUDA_DEVICE_NUM    = 0
18:11:32:CPAI_HALF_PRECISION     = Enable
18:11:32:CPAI_MODULE_SUPPORT_GPU = True
18:11:32:CUSTOM_MODELS_DIR       = %CURRENT_MODULE_PATH%/custom-models
18:11:32:MODELS_DIR              = %CURRENT_MODULE_PATH%/assets
18:11:32:MODEL_SIZE              = Medium
18:11:32:USE_CUDA                = True
18:11:32:YOLOv5_AUTOINSTALL      = false
18:11:32:YOLOv5_VERBOSE          = false
18:11:32:
[b]18:11:32:Started Object Detection (YOLOv5 6.2) module
18:11:33:detect_adapter.py: Traceback (most recent call last):
18:11:33:detect_adapter.py:   File "C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\detect_adapter.py", line 16, in 
18:11:33:detect_adapter.py:     from analysis.codeprojectai import CodeProjectAIRunner
18:11:33:detect_adapter.py:   File "../../SDK/Python\analysis\codeprojectai.py", line 36, in 
18:11:33:detect_adapter.py:     from analysis.requestdata import AIRequestData
18:11:33:detect_adapter.py:   File "../../SDK/Python\analysis\requestdata.py", line 8, in 
18:11:33:detect_adapter.py:     from PIL import Image
18:11:33:detect_adapter.py: ImportError: cannot import name 'Image' from 'PIL' (unknown location)
18:11:33:Module ObjectDetectionYolo has shutdown
18:11:33:detect_adapter.py: has exited
18:14:00:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...tectionYolo\detect_adapter.py"
18:14:00:
18:14:00:Module 'Object Detection (YOLOv5 6.2)' (ID: ObjectDetectionYolo)
18:14:00:Active:      True[/b]
18:14:00:GPU:         Support enabled
18:14:00:Parallelism: 0
18:14:00:Platforms:   all
18:14:00:FilePath:    detect_adapter.py
18:14:00:ModulePath:  ObjectDetectionYolo
18:14:00:Install:     PreInstalled
18:14:00:Runtime:     python37
18:14:00:Queue:       objectdetection_queue
18:14:00:Start pause: 1 sec
18:14:00:Valid:       True
18:14:00:Environment Variables
18:14:00:APPDIR                  = %CURRENT_MODULE_PATH%
18:14:00:CPAI_CUDA_DEVICE_NUM    = 0
18:14:00:CPAI_HALF_PRECISION     = Enable
18:14:00:CPAI_MODULE_SUPPORT_GPU = True
18:14:00:CUSTOM_MODELS_DIR       = %CURRENT_MODULE_PATH%/custom-models
18:14:00:MODELS_DIR              = %CURRENT_MODULE_PATH%/assets
18:14:00:MODEL_SIZE              = Medium
18:14:00:USE_CUDA                = True
18:14:00:YOLOv5_AUTOINSTALL      = false
18:14:00:YOLOv5_VERBOSE          = false
18:14:00:
18:14:00:Started Object Detection (YOLOv5 6.2) module
18:14:00:detect_adapter.py: Traceback (most recent call last):
18:14:00:detect_adapter.py:   File "C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\detect_adapter.py", line 16, in 
18:14:00:detect_adapter.py:     from analysis.codeprojectai import CodeProjectAIRunner
18:14:00:detect_adapter.py:   File "../../SDK/Python\analysis\codeprojectai.py", line 36, in 
18:14:00:detect_adapter.py:     from analysis.requestdata import AIRequestData
18:14:00:detect_adapter.py:   File "../../SDK/Python\analysis\requestdata.py", line 8, in 
18:14:00:detect_adapter.py:     from PIL import Image
18:14:00:detect_adapter.py: ImportError: cannot import name 'Image' from 'PIL' (unknown location)
18:14:00:Module ObjectDetectionYolo has shutdown
18:14:00:detect_adapter.py: has exited
18:14:35:Starting C:\Program Files...ws\python37\venv\scripts\Python "C:\Program Files...tectionYolo\detect_adapter.py"
18:14:35:
18:14:35:Module 'Object Detection (YOLOv5 6.2)' (ID: ObjectDetectionYolo)
18:14:35:Active:      True
18:14:35:GPU:         Support enabled
18:14:35:Parallelism: 0
18:14:35:Platforms:   all
18:14:35:FilePath:    detect_adapter.py
18:14:35:ModulePath:  ObjectDetectionYolo
18:14:35:Install:     PreInstalled
18:14:35:Runtime:     python37
18:14:35:Queue:       objectdetection_queue
18:14:35:Start pause: 1 sec
18:14:35:Valid:       True
18:14:35:Environment Variables
18:14:35:APPDIR                  = %CURRENT_MODULE_PATH%
18:14:35:CPAI_CUDA_DEVICE_NUM    = 0
18:14:35:CPAI_HALF_PRECISION     = Enable
18:14:35:CPAI_MODULE_SUPPORT_GPU = True
18:14:35:CUSTOM_MODELS_DIR       = %CURRENT_MODULE_PATH%/custom-models
18:14:35:MODELS_DIR              = %CURRENT_MODULE_PATH%/assets
18:14:35:MODEL_SIZE              = Medium
18:14:35:USE_CUDA                = True
18:14:35:YOLOv5_AUTOINSTALL      = false
18:14:35:YOLOv5_VERBOSE          = false
18:14:35:
18:14:35:Started Object Detection (YOLOv5 6.2) module
18:14:35:detect_adapter.py: Traceback (most recent call last):
18:14:35:detect_adapter.py:   File "C:\Program Files\CodeProject\AI\AnalysisLayer\ObjectDetectionYolo\detect_adapter.py", line 16, in 
18:14:35:detect_adapter.py:     from analysis.codeprojectai import CodeProjectAIRunner
18:14:35:detect_adapter.py:   File "../../SDK/Python\analysis\codeprojectai.py", line 36, in 
18:14:35:detect_adapter.py:     from analysis.requestdata import AIRequestData
18:14:35:detect_adapter.py:   File "../../SDK/Python\analysis\requestdata.py", line 8, in 
18:14:35:detect_adapter.py:     from PIL import Image
18:14:35:detect_adapter.py: ImportError: cannot import name 'Image' from 'PIL' (unknown location)
18:14:35:Module ObjectDetectionYolo has shutdown
18:14:35:detect_adapter.py: has exited

CPAI System Info tab

Code: Select all

Operating System: Windows (Microsoft Windows 10.0.19044)
CPUs:             1 CPU x 4 cores. 8 logical processors (x64)
GPU:              NVIDIA GeForce GTX 1080 Ti (11 GiB) (NVidia) 
                  Driver: 516.01 CUDA: 11.7 Compute: 6.1
System RAM:       16 GiB
Target:           Windows
BuildConfig:      Release
Execution Env:    Native
Runtime Env:      Production
.NET framework:   .NET 7.0.2
System GPU info:
  GPU 3D Usage       0%
  GPU RAM Usage      201 MiB
Video adapter info:
  Intel(R) HD Graphics 630:
    Adapter RAM        1,024 MiB
    Driver Version     31.0.101.2111
    Video Processor    Intel(R) HD Graphics Family
  NVIDIA GeForce GTX 1080 Ti:
    Adapter RAM        4 GiB
    Driver Version     31.0.15.1601
    Video Processor    NVIDIA GeForce GTX 1080 Ti
Global Environment variables:
  CPAI_APPROOTPATH = C:\Program Files\CodeProject\AI
  CPAI_PORT        = 32168
Blue Iris 5.x | Windows 10 Pro Slim | 16GB RAM | i7-7700 3.6 GHz | GTX 1080Ti FE | 8TB RAID NAS | 9 Cams | 2KVA UPS
PaulDaisy
Posts: 82
Joined: Mon Jan 16, 2023 5:06 pm

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by PaulDaisy » Fri Feb 03, 2023 6:42 pm

Interesting. I wonder if it has to do with "ImportError: cannot import name 'Image' from 'PIL' (unknown location)"
Are you able to process a screenshot in the AI Explorer? Does it detect anything? Does it say that it is using GPU while doing that?

The reason I ask is to see if the direct CPAI call works while the BI API call may not be working?
User avatar
Thixotropic
Posts: 891
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by Thixotropic » Fri Feb 03, 2023 10:02 pm

PaulDaisy wrote: Fri Feb 03, 2023 6:42 pm Interesting. I wonder if it has to do with "ImportError: cannot import name 'Image' from 'PIL' (unknown location)"
Are you able to process a screenshot in the AI Explorer? Does it detect anything? Does it say that it is using GPU while doing that?
Yes, that's what I saw that seemed wrong.

As for dropping a DAT file, maybe the BI AI Status tab shows something when a .dat is dropped on it, but I can't tell if this should be a detection event or not. It may be a nothing found event, but the BI window shows something, and a 15000ms timeout. I don't know if this is a legit test but it sort of looks like it's a canceled 'nothing found' event. I'll try to find one with a known event in it.
bi-status-2023-02-03_13-54-36.png
I don't have any luck at all in getting the AI Explorer to show anything. I've tried both Obect Detection and General Object Detection, neiither works. The image at the lower left of the AI Explorer shows a broken image and doesn't show the actual image at all (see below).

After a reboot when everything initially seems to be working, the GPU is seen in BI's status line as being used, but after almost exactly 5 minutes it disappears and never returns.

I'm pretty sure that even during that 5-minute 'seems to be working' period, it's not actually working.

I've tried restarting the AI through BI and through the CodeProject AI web interface, and nothing happens as far as I can tell. CPAI cycles off and then back on, but the GPU is out of the loop and nothing is being detected.

About the only thing coming from the AI are error messages indicating a 15000ms timeout (which is the 15 secs it's set to in BI's AI tab). Also, in the AI Explorer "Custom Obkect Detection" section, the dropdown does not show IPCam-Combined as a selection, but it shows

General IP-Cam
License Plates v2
Open Logo
Human (Actionet) v2

It doesn't show the ipcam-combined at all. I don't know what to make of that.

-
When the system starts, everything initially looks fine, but it's not. The AI web panel says it's online but the AI Explorer window says it's offline, then maybe blinks when I try to analyze a file, but goes back to offline status immediately.

At this point I have no idea what to try or to look for, lol.
Attachments
startup-2023-02-03_11-30-27.png
Last edited by Thixotropic on Sat Feb 04, 2023 1:10 am, edited 2 times in total.
Blue Iris 5.x | Windows 10 Pro Slim | 16GB RAM | i7-7700 3.6 GHz | GTX 1080Ti FE | 8TB RAID NAS | 9 Cams | 2KVA UPS
PaulDaisy
Posts: 82
Joined: Mon Jan 16, 2023 5:06 pm

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by PaulDaisy » Fri Feb 03, 2023 10:16 pm

Thixotropic wrote: Fri Feb 03, 2023 10:02 pm I don't have any luck at all in getting the AI Explorer to show anything. I've tried both Obect Detection and General Object Detection, neiither works. The image at the lower left of the AI Explorer shows a broken image and doesn't show the actual image at all (see below).
...
When the system starts, everything initially looks fine, but it's not. The AI web panel says it's online but the AI Explorer window says it's offline, then maybe blinks when I try to analyze a file, but goes back to offline status immediately.
I have a suspicion that something is wrong with the AI installation. Are you using the latest one (2.0.7)? I didn't dare installing it and my 1.6.8 is chugging along happily. If I suspect something with AI I always drop an image into Explorer and it detects things, telling me the AI server at least is set up correctly. If it is not, then the BI API will time out as you are seeing.
This is what I see in Explorer. Granted, the garden light is not a "person", there is room for model improvement there, but the garbage truck is definitely a truck.
Also, the fact that your image in Explorer appears "broken" and there is that error regarding an "unknown location" above... maybe something didn't work right during install? Maybe the PATH variable didn't update via script (a wild guess)?..
ScreenShot011.jpg
User avatar
Thixotropic
Posts: 891
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by Thixotropic » Fri Feb 03, 2023 11:18 pm

PaulDaisy wrote: Fri Feb 03, 2023 10:16 pm I have a suspicion that something is wrong with the AI installation. Are you using the latest one (2.0.7)? I didn't dare installing it and my 1.6.8 is chugging along happily. If I suspect something with AI I always drop an image into Explorer and it detects things, telling me the AI server at least is set up correctly. If it is not, then the BI API will time out as you are seeing.

Also, the fact that your image in Explorer appears "broken" and there is that error regarding an "unknown location" above... maybe something didn't work right during install? Maybe the PATH variable didn't update via script (a wild guess)?..
ScreenShot011.jpg
I think first I'll try the 'Repair' option in CPAI, and if that doesn't work then reverting to CPAI 1.68 seems like a reasonable test; let me give that a go and I'll see what happens.
Blue Iris 5.x | Windows 10 Pro Slim | 16GB RAM | i7-7700 3.6 GHz | GTX 1080Ti FE | 8TB RAID NAS | 9 Cams | 2KVA UPS
User avatar
Thixotropic
Posts: 891
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by Thixotropic » Sat Feb 04, 2023 5:23 am

Okay, so after a fair bit of testing, here's what I've found...

1) The disappearing GPU indicator in the bottom status bar is *probably* just a minor display bug in BI. Even though the GPU status indicator disappears, I *think* the GPU is still actually being used. The reason I suspect this is described below.

2) BI 5.6.9.7 and CPAI 2.06/2.07 appear to have some real problems playing together on my BI box. Whatever it is, it's definitely not a viable combination on my PC (YMMV).

The weird problems I saw are too numerous and too nebulous to pin down and describe properly. Let's just say on my PC it was a 100% repeatable car crash. Again, this is just my PC, that combination might work perfectly on your box.

I rolled back to BI 5.6.8.4, but CPAI 2.07 still seemed to end up failing with various errors, many related to some file in the ObjectDetectionYolo directory or one of its dependents. It would exit and die, sometimes it would restart, sometimes it wouldn't or couldn't.

Even though the AI Explorer and other CPAI displays would show the AI server was online, it could not or would not communicate with BI and would end up timing out. It also did some very weird, spooky stuff when trying to start or stop CPAI and whatever it was doing, it was always fatal.

I tried the 'Repair' option for CPAI but it didn't make any difference. So, I uninstalled CPAI 2.07, reinstalled CPAI 1.68, and rebooted.

The system came up and so far seems to be working for the most part. I'm not getting quite the detection times I'd hoped for yet, but it's doing car and person recognition with detection times typically running between 110ms and 150ms (a few higher, a few lower, but most fall within that range).

The GPU indicator is AWOL but the card seems to be being used based on the detection times (??), so maybe it's just a display issue and not actually something with the card or CPAI.

I have for the moment left all the default models in place, but I plan to remove the unneeded ones next and see if that gives me any better detection times.

Recap

So, BI 5.6.8.4 and CPAI 1.68 seem like a viable combination, at least on this PC, whereas BI 5.6.9.7 and CPAI 2.06/2.07 are definitely not.

I notice that BI 5.6.9.8 just came out. Perhaps the disappearing GPU issue is fixed and/or a bug fix for the CPAI 2.07 compatibility? One can hope.

So I'll try the BI 5.6.9.8 upgrade, and if everything still works after that, try upgrading CPAI to 2.07. But it's late, so I'll give it a shot tomorrow.
Blue Iris 5.x | Windows 10 Pro Slim | 16GB RAM | i7-7700 3.6 GHz | GTX 1080Ti FE | 8TB RAID NAS | 9 Cams | 2KVA UPS
rdxny
Posts: 36
Joined: Tue Oct 18, 2022 12:01 am

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by rdxny » Sat Feb 04, 2023 3:15 pm

Thixotropic wrote: Sat Feb 04, 2023 5:23 am
1) The disappearing GPU indicator in the bottom status bar is *probably* just a minor display bug in BI. Even though the GPU status indicator disappears, I *think* the GPU is still actually being used. The reason I suspect this is described below.
I also have the disappearing GPU on the console bottom, but if I look at my GPU status in windows task manager, I get peaks in GPU usage when cameras get triggered in BI, so it looks like a display bug in BI only. (using .NET in CPAI with built in GPU )
User avatar
Thixotropic
Posts: 891
Joined: Wed Sep 04, 2019 7:20 pm
Location: Low-Earth Orbit

Re: Upgrading to a GTX 1080 Ti - any tips?

Post by Thixotropic » Sat Feb 04, 2023 4:45 pm

rdxny wrote: Sat Feb 04, 2023 3:15 pm I also have the disappearing GPU on the console bottom, but if I look at my GPU status in windows task manager, I get peaks in GPU usage when cameras get triggered in BI, so it looks like a display bug in BI only. (using .NET in CPAI with built in GPU)
Hmmm, I'm not seeing any activity in my GPU monitor, it's flat as a board even though it identifies the GPU as present. So, maybe it's not being used...?
-
gpu-2023-02-04_8-44-18.png
Blue Iris 5.x | Windows 10 Pro Slim | 16GB RAM | i7-7700 3.6 GHz | GTX 1080Ti FE | 8TB RAID NAS | 9 Cams | 2KVA UPS
Post Reply