Friday, November 20, 2026

Agama Unattended installation remote Tumbleweed KVM Guests via multi-threaded Agama-ProxyV2.service on top of Tumbleweed Server

Optimal Thread Performance via Tumbleweed's Upstream Kernel
When your Python-based web server spawns a native OS thread to handle each incoming GET ./profile.json request: * Efficient Futex & GIL Release: Tumbleweed delivers the latest vanilla Linux kernels.These upstream kernels feature highly optimized scheduler implementations and fast user-space locking. Because your threads are primarily handling I/O operations (fetching  
profile.json and sending it over network sockets),the Python Global  
Interpreter Lock (GIL) is immediately released during network waits. The latest kernel maximizes the scheduling efficiency of these  
released threads across your  raw CPU cores. * Direct Page Cache Access: On a bare-metal server, the first  
thread reading profile.json pulls it from disk.Linux immediately  
retains that file in the bare-metal page cache (RAM). Subsequent  
concurrent threads spawned by incoming requests serve the file  
directly out of host memory at native bus speeds. * Zero Dependency Overhead: Operating directly on your unique Tumbleweed instance allows the script (as agama-proxyV2)  
to run out-of-the-box using python's built-in http.server and  
socketserver modules. There is no need to compile complex  
wheels or maintain external framework packages. * Native Network Stack Performance: Serving profiles straight  
to physical Network Interface Cards (NICs) ensures that  
your multi-threaded server responds at maximum physical  
line rate. It avoids the latency overhead associated with virtual  
network bridges or packet replication typical inside guest OS  
environments.
==================================================
Multithreaded version of agama-proxyV2.py (*)
=========================================
# cat agama-proxyV2.py
import http.server
import socketserver
import json
import os

# Set the path to where your developers drop the profile.json file
PROFILE_DIR = "/srv/agama"
try:
    os.chdir(PROFILE_DIR)
except Exception as e:
    print(f"[Initialization Error] Could not change directory to {PROFILE_DIR}: {e}")

class Agama25Transformer(http.server.SimpleHTTPRequestHandler):

    def address_string(self):
        """Return the raw IP address directly, bypassing slow reverse DNS lookups."""
        return self.client_address[0]

    def do_GET(self):
        if self.path == "/profile.json":
            try:
                # Load the raw file exported by your developers or Web UI
                with open("profile.json", "r", encoding="utf-8") as f:
                    data = json.load(f)

                # Check if "user" key exists and matches the problematic array layout
                if "user" in data:
                    raw_user = data["user"]

                    if isinstance(raw_user, list) and len(raw_user) > 0:
                        print("[Proxy Engine] Found user array block. Flattening structure for Agama 25.0.0...")
                        data["user"] = raw_user[0]

                    elif isinstance(raw_user, dict) and "users" in raw_user:
                        print("[Proxy Engine] Found nested users dictionary block. Unwrapping entries...")
                        if isinstance(raw_user["users"], list) and len(raw_user["users"]) > 0:
                            data["user"] = raw_user["users"][0]

                # Stream the newly formatted JSON bytes natively
                response_bytes = json.dumps(data).encode("utf-8")

                self.send_response(200)
                self.send_header("Content-Type", "application/json; charset=utf-8")
                self.send_header("Content-Length", str(len(response_bytes)))
                self.end_headers()
                self.wfile.write(response_bytes)
                return
            except Exception as e:
                print(f"[Proxy Error] Intercept optimization failed: {e}")
                # Fallback gracefully instead of crashing the thread completely
                self.send_error(500, f"Internal Proxy Error: {e}")
                return

        super().do_GET()

# Enterprise Change: Use ThreadingTCPServer instead of a blocking TCPServer
class ThreadingHTTPServer(socketserver.ThreadingMixIn, socketserver.TCPServer):
    # Ensure threads are cleaned up immediately by the OS when they finish
    daemon_threads = True

if __name__ == "__main__":
    ThreadingHTTPServer.allow_reuse_address = True

    # Bind to port 8080. Each incoming Agama connection gets its own isolated thread.
    with ThreadingHTTPServer(("0.0.0.0", 8080), Agama25Transformer) as httpd:
        print("[Enterprise Engine] Agama 25 Concurrent Deployment Proxy active on port 8080...")
        try:
            httpd.serve_forever()
        except KeyboardInterrupt:
            print("\nShutting down server gracefully.")

===============================================

Creating service unit Agama-proxyV2 Service on top Tumbleweed Server bare metal (*)

===============================================

[Unit]
Description=Agama 25 Multi-Threaded Deployment Proxy Daemon
After=network-online.target local-fs.target remote-fs.target
Wants=network-online.target

[Service]
Type=simple
User=boris
WorkingDirectory=/home/boris/Downloads
Environment=PYTHONUNBUFFERED=1

# Raise the start window to 2 minutes so the 90s safety sleep #never triggers a systemd timeout 

TimeoutStartSec=120s

# Bypasses the 90s sleep if you are just restarting the service manually
ExecStartPre=/bin/sh -c 'if [ "$(systemctl is-system-running)" = "initializing" ] || [ "$(systemctl is-system-running)" = "starting" ]; then sleep 90; fi'

# System python runtime
ExecStart=/usr/bin/python3 ./agama-proxyV2.py

Restart=on-failure
RestartSec=5s

# Force systemd to wipe out all threads and free up the socket descriptor instantly
KillMode=control-group
SendSIGKILL=yes

StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target
 

==================================

Next step followed by system reboot

================================== 

sudo systemctl daemon-reload
sudo systemctl reset-failed agama-proxyV2.service
sudo systemctl enable --now agama-proxyV2.service
 

=========================================

Log into rebooted Tumbleweed instance on bare metal 

=========================================

Wait until you service  agama-proxyV2 would be activated

localhost:~ # sudo ss -tulpn | grep 8080
tcp   LISTEN 0      5                                 0.0.0.0:8080      0.0.0.0:*    users:(("python3",pid=3532,fd=3))

localhost:~ # ps -ef|grep 3532
boris       3532       1  0 15:34 ?        00:00:00 /usr/bin/python3 ./agama-proxyV2.py

Yours PID might be different from 3532

REFERENCES

(*) Assisted by Google AI. 

No comments:

Post a Comment