run unsigned binaries on MacOS

 I was not sure if unsigned binaries would run on MacOS, and whether one can override its behaviour. So, apparently MacOS 10.15 Sonoma onwards, unsigned binaries won't run by default, but we can specifically white-list any binary we want to run, and then it will run.

https://github.molgen.mpg.de/pages/bs/macOSnotes/mac/mac_procs_unsigned.html

https://discussions.apple.com/thread/254786001?sortBy=rank

Either via the GUI - 

* Try to open myFancyBinary in Finder. This will fail.

*Open System Preferences, choose the Security control panel, select the General tab.

*Look for the message: “myFancyBinary was blocked from opening because it is not from an identified developer.”

*Click the Open Anyway button to the right of the message.

or via the terminal - 

spctl --add /path/to/myBinary

And for getting the full path, and easy way is to drag and drop the binary from Finder into the Terminal window.

Stable Diffusion tests and benchmarks

I had tried out Stable Diffusion on Windows last year, now I wanted to try it out on Linux and if possible, using Docker. 

Unfortunately, the docker method failed. Searching for stable diffusion docker with web ui

https://sygil-dev.github.io/sygil-webui/docs/Installation/docker-guide/

Stable Diffusion in Docker | Open Source AI Image Generation Self-Hosted

talks about

https://github.com/AbdBarho/stable-diffusion-webui-docker

https://github.com/AbdBarho/stable-diffusion-webui-docker/wiki/Setup

(I had to run everything with sudo prefix.)

sudo docker compose --profile download up --build

roughly 12GB of data to be downloaded.

Unfortunately, after 2 hours of downloading, using the Automatic1111 profile,

sudo docker compose --profile auto up --build

Error response from daemon: could not select device driver "nvidia" with capabilities: [[compute utility]]

So, probably I need --profile auto-cpu.

But that gave an error,

File "/stable-diffusion-webui/webui.py", line 13, in <module>

etc

ImportError: cannot import name 'TypeIs' from 'typing_extensions' (/opt/conda/lib/python3.10/site-packages/typing_extensions.py)

Deleting the dir, cloning the repo and starting again - again downloading 12 GB, 

And again failed.

sudo docker compose --profile auto-cpu up --build

Mounted styles.csv

auto-cpu-1  | Mounted ui-config.json

auto-cpu-1  | mkdir: created directory '/data/config/auto/extensions'

auto-cpu-1  | Mounted extensions

auto-cpu-1  | Installing extension dependencies (if any)

auto-cpu-1  | Traceback (most recent call last):

auto-cpu-1  |   File "/stable-diffusion-webui/webui.py", line 13, in <module>

auto-cpu-1  |     initialize.imports()

auto-cpu-1  |   File "/stable-diffusion-webui/modules/initialize.py", line 23, in imports

auto-cpu-1  |     import gradio  # noqa: F401

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/gradio/__init__.py", line 3, in <module>

auto-cpu-1  |     import gradio.components as components

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/gradio/components/__init__.py", line 3, in <module>

auto-cpu-1  |     from gradio.components.bar_plot import BarPlot

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/gradio/components/bar_plot.py", line 7, in <module>

auto-cpu-1  |     import altair as alt

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/__init__.py", line 649, in <module>

auto-cpu-1  |     from altair.vegalite import *

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/vegalite/__init__.py", line 2, in <module>

auto-cpu-1  |     from .v5 import *

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/vegalite/v5/__init__.py", line 2, in <module>

auto-cpu-1  |     from altair.expr.core import datum

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/expr/__init__.py", line 11, in <module>

auto-cpu-1  |     from altair.expr.core import ConstExpression, FunctionExpression

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/expr/core.py", line 6, in <module>

auto-cpu-1  |     from altair.utils import SchemaBase

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/utils/__init__.py", line 14, in <module>

auto-cpu-1  |     from .plugin_registry import PluginRegistry

auto-cpu-1  |   File "/opt/conda/lib/python3.10/site-packages/altair/utils/plugin_registry.py", line 13, in <module>

auto-cpu-1  |     from typing_extensions import TypeIs

auto-cpu-1  | ImportError: cannot import name 'TypeIs' from 'typing_extensions' (/opt/conda/lib/python3.10/site-packages/typing_extensions.py)

auto-cpu-1 exited with code 1

Then I thought I would try chainner, https://chainner.app/download

gives a deb file, for ease of installation. It has a dependency manager also. 

But the dependency manager does not install stable diffusion! So, trying

https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-NVidia-GPUs#linux

(

current python version is 3.11.5 on this Linux mint based on ubuntu 24.04 - which may run into problems,

https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/9473

- we're supposed to run on python 3.10-venv

)

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB. GPU 0 has a total capacty of 3.94 GiB of which 31.50 MiB is free. Including non-PyTorch memory, this process has 3.73 GiB memory in use. Of the allocated memory 3.60 GiB is allocated by PyTorch, and 77.43 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Stable diffusion model failed to load

Can try lowvram command, since we have the NVidia 1060 -  

https://techtactician.com/stable-diffusion-low-vram-memory-errors-fix/

export COMMANDLINE_ARGS="--lowvram"

./webui.sh

Success. Does a 512x512 generate in ~ 1 minute, 4x ESRGAN upscale resize in ~30 sec.

https://github.com/AUTOMATIC1111/stable-diffusion-webui

this has a listing of how to make prompts.

My primary use case was for upscaling. I took a picture which I had clicked with the mobile camera in its "50 megapixel" mode, which shows quite a bit of "softening" when viewed at 100% - then cropped to 4096x4096, scaled down to 512x512 (1k images were causing out of VRAM errors) and then scaled up to 4096x4096. Interesting to see the results for a jpg show lots of ringing, while the results for a png seems to make things sharper. 

A portion of jpg input on the left, with output on the right, processed as 

scale down to 512 px, save as jpg, and then upscale to 4x - this shows ringing. Click on the image to view larger size.

A portion of jpg input on the left, with output on the right, processed as 

scale down to 512 px, save as png, and then upscale to 4096 - this shows an interesting pointillistic effect, and has made edges if the image sharper - clearly seen in the foreground trees.  Click on the image to view larger size.

I'll write a separate post on Easy Diffusion, since this post has already become quite long.

experiments with LLMs and docker

Tried out some "easy to use locally" Large Language Models (LLMs) with the motivation of being able to use them with our own documents. 

Looking for easy ways to install and run them, via Run Local LLMs on Hardware from $50 to $50,000 - We Test and Compare!

https://ollama.com/download

ollama list

ollama pull llama3.1:latest (~4 GB - the download went at nearly 4 MBps on the Airtel 40 Mbps connection, and around 6 MBps on Airtel 5G - 12 minutes)

ollama pull llama3.2:latest (~2 GB, and runs faster on low-end hardware)

(run as ollama run llama3.1:latest --verbose to get tokens/sec.

)

With 3.1, it was going at ~4 tokens per second on the desktop. CPUs, I think. With llama3.2, it was going at 23 tokens / sec - quite an acceptable speed. On the Windows laptop, running on CPU, llama3.2 at 9 tokens / sec.

For a web ui, via Set up a Local AI like ChatGPT on your own machine!

Wanted to try the docker route - installed docker via https://docs.docker.com/engine/install/ubuntu/ by setting up docker's repository. All my docker commands needed to be prefaced by sudo, since I didn't go through the post-install steps.

Trying to run open-webui with the commandline given in the youtube video, 

sudo docker run -d -p 3000:8080 --gpus=all -v ollama:/root/.ollama -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:ollama 

failed. https://docs.openwebui.com/getting-started/quick-start

has the steps.

sudo docker pull ghcr.io/open-webui/open-webui:main

For Nvidia GPU support, add --gpus all to the docker run command:

Single user mode - 

sudo docker run -d -p 3000:8080 --gpus all -e WEBUI_AUTH=False -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

That also failed. "Unable to load models". Then, found that we need to start ollama separately in another terminal window, from How to Install Ollama, Docker, and Open WebUI on Windows - YouTube
and we need a different set of parameters in the command line if ollama is already installed on our system. So, got the openwebui to work with two modifications:

ran from administrator cmd (on Windows)

without -d (so that I can see the errors if any - do not detach)

and must start ollama first in another window with

ollama run llama3.2

 docker run  -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

But connecting stable diffusion to open-webui seems to be complicated - 

https://torgeir.dev/2024/05/connecting-stable-diffusion-webui-to-your-locally-running-open-webui/

So, will run Stable Diffusion (for image generation and upscaling) from its own web ui instead.

set up app password for sending emails from server

One of our wordpress servers was to be used to send some emails to users. So, changed the ssmtp "from" account from our internal server's google account to another one, meant to be used for this server. 

• Created a new google workspace account
• Logged in to it for the first time
• Set up recovery email, recovery phone, authenticator, enabled two-factor authentication
• Created an app password
• edited the /etc/ssmtp/ssmtp.conf with the new userid and app password.

rclone - GUI and remote server-side copy possibilities

 Tried out the GUI at https://rcloneview.com/src/download.html using the Linux AppImage. The remote already configured for my user account was detected, but the directory listing didn't seem to work. Neither did the "Mount". 

Then, server-side copy possibilities - when doing remote to remote copies, the local network is used - so that's not really useful. And on the same remote, a server-side copy is done using hard links - https://forum.rclone.org/t/sftp-remote-server-side-copy/41867 - so that's a bit iffy. Server side copy might be useful for copying between two different cloud providers. The fastest option for our SFTP remotes would be to ssh into one of the machines, install rclone there, create an sftp remote for the other machine, and do the copy from the ssh shell - that would be much faster than bringing all the data to our local network and sending it back.

compiling a wxwidgets project with cmake on Windows

Run into issues building the simple hello world example from wxwidgets using cmake on Windows. On Linux, the default github cmake action did not need much modification. 

CMakeLists.txt

Linux cmake build workflow

But for building on Windows, we needed to add some env variables in the workflow

env:

  # Customize the CMake build type here (Release, Debug, RelWithDebInfo, etc.)

  BUILD_TYPE: Release

  WXWIN: C:/wxWidgets-3.2.6

  wxWidgets_ROOT_DIR: C:/wxWidgets-3.2.6

  wxWidgets_LIBRARIES: C:/wxWidgets-3.2.6/include

  wxWidgets_INCLUDE_DIRS: C:/wxWidgets-3.2.6/lib/vc14x_x64_dll

  ARCH: x64

or else the find_package would complain about not finding wxWidgets_LIBRARIES and wxWidgets_INCLUDE_DIRS

Also, 

cmake -G"Visual Studio 17 2022" -B ${{github.workspace}}/build -DCMAKE_BUILD_TYPE=${{env.BUILD_TYPE}}

to specify which version of Visual Studio to use - and for this version, the default toolset is 14.3 which is the one used to build this set of wxwidgets binaries.
https://docs.wxwidgets.org/3.2.1/plat_msw_binaries.html

But we also had to add a dummy main() to get rid of a linker error, but that breaks the Linux build. 

/home/runner/work/wxOCVWarp/wxOCVWarp/hello.cpp:78:5: error: conflicting declaration of C function ‘int main()’

   78 | int main() { return 0;}

      |     ^~~~

In file included from /usr/include/wx-3.0/wx/wx.h:25,

                 from /home/runner/work/wxOCVWarp/wxOCVWarp/hello.cpp:3:

/home/runner/work/wxOCVWarp/wxOCVWarp/hello.cpp:11:1: note: previous declaration ‘int main(int, char**)’

   11 | wxIMPLEMENT_APP(MyApp);

      | ^~~~~~~~~~~~~~~

gmake[2]: *** [CMakeFiles/OCVWarp.dir/build.make:79: CMakeFiles/OCVWarp.dir/hello.cpp.o] Error 1

gmake[1]: *** [CMakeFiles/Makefile2:87: CMakeFiles/OCVWarp.dir/all] Error 2

gmake: *** [Makefile:91: all] Error 2

So, wrapped an ifdef around the dummy main(), and all is well.

#ifdef _WIN64

int main() { return 0;}

#endif

rclone over ssh / sftp

 Tried installing the latest rclone - version v1.69.1 using the deb file and testing an ssh / sftp remote. Finally got it to work. Used the key_file option and passed on the path to our AWS-generated PEM file as the key_file - all other options were left as default with <ENTER>

A point to note is that if we do 

rclone ls nameofourremote:/

it will list all the files on the remote recursively! Instead, we can do

rclone lsd nameofourremote:/some/path

to list only the directories inside the specified path.

setting up a server with a dotnet api

A subset of the previous post. Steps carried out - 

- created a fresh user using adduser

- MySQL database was already running.

- created the subdomains for website and API

- created the appropriate virtual servers for the above.

- copied the uat files to prod

- created a service running on a port like 5000 or 5010 etc for the dot net api at /etc/systemd/system/ourapi.service

systemctl start ourapi

systemctl enable ourapi

(Once the team changes the files referred to by the api service , I will need to restart the service.)

- tested api and website url

Large Language Models with our own documents - RAG for LLM

RAG = Retrieval Augmented Generation - can be done locally as well as on cloud - explanatory video - 

Feed Your OWN Documents to a Local Large Language Model!

In the video, openwebui is used to run the local LLM - 

https://docs.openwebui.com/features/

If using docker (for ease of setup), and we want to copy our documents to the LLM's docker container, Docker copy command - 

Copying Files to and from Docker Containers - GeeksforGeeks

docker container cp source_local container:/path

sudo docker cp ~/Desktop/to-be-copied.txt 135950565ad8:/to-be-copied.txt

(the container id like 135950565ad8 above, is usually seen as a root@containerid prompt in the running container)

And apparently in the latest versions of openwebui, there is integration inside the UI itself for uploading documents - https://docs.openwebui.com/features/rag#local-and-remote-rag-integration

"Local documents must first be uploaded via the Documents section of the Workspace area to access them using the # symbol before a query. Click on the formatted URL in the that appears above the chat box. Once selected, a document icon appears above Send a message, indicating successful retrieval."

See this post for some initial setup, experiments and benchmarks - https://hnsws.blogspot.com/2025/04/experiments-with-llms-and-docker.html