Thursday, December 23, 2021

Uploading dictionaries into SQLite tables via Pandas DataFrames on Python 3.9.9

 The following below is a not entirely trivial algorithm for students that creates a Python dictionary after scanning text according to the conditions of the problem, which gets converted into Pandas Dataframe and uploaded to SQLite database table to be analyzed via nested SQL Query.

Problem itself

A text file (beta-wide.txt) consists of many lines, each of which can contain letters from the set A, B, C, D, E, F. For the entire file, you need to determine which letter is most often forms substrings of maximum length. As an answer, you must enter a string consisting of the found letters, alphabetically without delimiters

(.env) [boris@fedora34server WORK]$ ll beta-wide.txt

-rw-rw-r--. 1 boris boris 59420 Dec 23 16:20 beta-wide.txt














Python code building dictionary to be uploaded into SQLite table

via pandas DataFrame

import pandas as pd

import sqlite3

connection = sqlite3.connect('data_pandas.db')

c = connection.cursor()

c.execute('CREATE TABLE IF NOT EXISTS results(symv_name text, symv_qnt number)')

connection.commit()

seqLetter = []

file = open('./beta-wide.txt')

for line in file:

    currLetrs = set()

    curLen = 0

    maxLen = 1

    prev = ' '

    for next in line:

        if prev == next:

            curLen +=1

            if curLen > maxLen:

                maxLen = curLen

                currLetrs = set([next])

            elif maxLen == curLen:

                currLetrs.add(next)

        else:

          curLen =1

        prev = next

    seqLetter = seqLetter + list(currLetrs)

d = {}

for i in set(seqLetter):

    key = i

    d[key] = d[key] = d.get(key,0) + seqLetter.count(i)

data_items = d.items()

data_list = list(data_items)

df = pd.DataFrame(data_list, columns= ['symv_name','symv_qnt'])

df.to_sql('results', connection, if_exists='replace', index = False)

query = "SELECT * FROM results  WHERE symv_qnt = (SELECT max(symv_qnt) FROM results)"

data = pd.read_sql_query(query,connection)

print(data)

SQLiteBrowser double check






































Monday, December 13, 2021

Exersing Python 3.10 on Fedora 25 via sorting Pandas DataFrame

 The following below is a not entirely trivial algorithm for students that creates a Python dictionary when scanning text according to the conditions of the problem, which is then processed as Pandas Dataframe to provide the output required

The original problem itself

The text file alpha-wide.txt contains only capital letters of the Latin alphabet (ABC… Z). Identify the character that most often occurs in the file immediately after the letter X. In the answer first write down this character, and then separate by blank how many times it occurred after letters X. If there are several such characters, you need to display the one that appears earlier in the alphabet.

The file alpha-wide.txt contains a pretty long string

Python code has been set up in a virtual environment with pandas installed via pip

import pandas as pd

with open("./alpha-wide.txt") as F:

  string = F.readline() # getting next line

d={} # declare dictionary of letters 

while "X" in string:

    n = string.find("X") # index "X"

    key = string[n+1]    # create key = Letter

    d[key] = d.get(key,0)+1

    string = string[n+1:] # slice, form a new line

# print(d)

data_items = d.items()

data_list = list(data_items)

df = pd.DataFrame(data_list,columns = ['X1','Y1'])

df.sort_values("Y1", axis = 0, ascending = True, inplace = True, na_position ='last')

print("DataFrame has been sorted by Y1")

print(df)





Sunday, December 12, 2021

Exerising Python 3.10 on Fedora 25 via sorting Pandas DataFrame

The following below is a not entirely trivial algorithm for students that creates a Python dictionary when scanning text according to the conditions of the problem, which is then processed as Pandas Dataframe to provide the output required

The original problem itself

The text file alpha-wide.txt contains only capital letters of the Latin alphabet (ABC… Z). Identify the character that most often occurs in the file immediately after the letter X. In the answer first write down this character, and then separate by blank how many times it occurred after letters X. If there are several such characters, you need to display the one that appears earlier in the alphabet.

The file alpha-wide.txt contains a pretty long string












Python code has been set up in a virtual environment with pandas installed via pip

import pandas as pd

with open("./alpha-wide.txt") as F:

  string = F.readline() # getting next line

d={} # declare dictionary of letters 

while "X" in string:

    n = string.find("X") # index "X"

    key = string[n+1]    # create key = Letter

    d[key] = d.get(key,0)+1

    string = string[n+1:] # slice, form a new line

# print(d)

data_items = d.items()

data_list = list(data_items)

df = pd.DataFrame(data_list,columns = ['X1','Y1'])

df.sort_values("Y1", axis = 0, ascending = True, inplace = True, na_position ='last')

print("DataFrame has been sorted by Y1")

print(df)



System information


Tuesday, October 12, 2021

Python Wrapper to find all primes from a given interval via sieve of Eratosthenes released as C++ procedure

 We intend to rely on https://github.com/mathbunnyru/python-cpp-extension . We also would to convert lists to sets and vice-versa and utilize method set1.difference(set2) . Sometimes Python tricks make you really excited 

Download and extract tarball on Fedora 34 with "C Development tools" and python-devel installed . Setup python venv in your working directory

 $ python3 -m venv .env

 $ source .env/bin/activate

and run straight away:-

$ python setup.py install


(.env) [boris@fedora34server python-cpp-extension-delta]$ cat deltaBetween.py

import cpp_python_extension
uplimit,downlimit =0,0
znBig = []
znSmall = []

uplimit = int(input("Input upper bound :"))
downlimit = int(input("Input lower bound :"))

znBig = cpp_python_extension.sieve_of_eratosthenes(uplimit)
znSmall = cpp_python_extension.sieve_of_eratosthenes(downlimit)

setSmall = set(znSmall)
setBig = set(znBig)

deltaSet = setBig.difference(setSmall)
deltaList = list(deltaSet)
deltaList.sort()

print(deltaList)


Sunday, October 10, 2021

Python C++ Wrapper for another YandexQ question via sieve of eratosthenes released as C++ procedure

This post is an immediate follow up for http://lxer.com/module/newswire/view/306255/index.html  what will result significant performance boost vs python's version of the sieve of Eratosthenes procedure.

We intend to rely on https://github.com/mathbunnyru/python-cpp-extension . Download and extract tarball on Fedora 34 with "C Development tools" and python-devel installed . Setup python venv in your working directory

 $ python3 -m venv .env

 $ source .env/bin/activate

and run straight away:-

$ python setup.py install

(.env) [boris@fedora34server python-cpp-extension-delta]$ ll

total 36

drwxrwxr-x. 5 boris boris   89 Oct 10 09:37 build

drwxrwxr-x. 2 boris boris   90 Oct 10 09:37 cpp_python_extension.egg-info

-rw-rw-r--. 1 boris boris  751 Oct 10 11:57 delta16.py

-rw-rw-r--. 1 boris boris  414 Oct 10 12:01 deltaBetween.py

drwxrwxr-x. 2 boris boris   61 Oct 10 09:37 dist

-rw-rw-r--. 1 boris boris   85 Oct 10 09:18 MyProg.py

-rw-rw-r--. 1 boris boris 1192 Oct 10 09:18 README.md

-rw-rw-r--. 1 boris boris  682 Oct 10 09:18 setup.py

-rw-rw-r--. 1 boris boris  450 Oct 10 09:18 sieve.cpp

-rw-rw-r--. 1 boris boris  140 Oct 10 09:18 sieve.h

-rw-rw-r--. 1 boris boris 1542 Oct 10 09:18 sieve_python_wrapper.cpp

-rw-rw-r--. 1 boris boris  200 Oct 10 09:18 test_benchmark.py

(.env) [boris@fedora34server python-cpp-extension-delta]$

Now update delta16.py from post  http://lxer.com/module/newswire/view/306255/index.html

Then update delta16.py as follows

(.env) [boris@fedora34server python-cpp-extension-delta]$ cat delta16.py

import cpp_python_extension

def sumDigits(N,k):

    sum = 0

    while (N != 0):

        sum = sum + N % k

        N = N // k

    return sum

def SearchNumber(uplimit,sdg):

   imax = 0

   zn = []

   # Here we invoke cpp_python_extension.sieve_of_eratosthenes 

   # been written in C++ via Python C API which returns python's 

   # object list containing all prime numbers <= uplimit

   zn = cpp_python_extension.sieve_of_eratosthenes(uplimit) 

   spisok = set(zn)

   for i in range(2,uplimit):

        # Look for highest prime number <= uplimit 

        # and summa of digits <= sdg

        if (i in spisok)  and sumDigits(i,10) <= sdg:

            if i > imax:

                 imax = i

   print("Maximum = ",imax)

def main():

   uplimit,sdg = 0,0

   uplimit = int(input("Input range of scan :"))

   sdg = int(input("Input sum of digits :"))

   SearchNumber(uplimit,sdg) 

if __name__ == "__main__":

   main()




















Saturday, October 9, 2021

Solution another YandexQ question via sieve of eratosthenes in Python

 Problem itself 

What is the maximum natural number X satisfies the condition: X - prime AND NOT X> 1000 AND the sum of the digits X ≤ 25 ?

The major target of this approach is performance improvement when looking for the highest prime number less than 10.000.000 with a given sum of decimal digits. In general, the problem  requires  implementation of sieve of Eratosthenes

Python code implementing  sieve of Eratosthenes

[boris@fedora34server YANDEXQ]$ cat delta16.py

def sumDigits(N,k):

    sum = 0

    while (N != 0):

        sum = sum + N % k

        N = N // k

    return sum

def eratosphen(n,a):

   prime = [True for i in range(n + 1)]

   prime[0] = prime[1] = False

   i = 2

   while i * i <= n:

        if prime[i]: 

            j = i * i

            while j <= n:

                prime[j] = False

                j += i

        i += 1

   for i in range(2, n + 1):

         if prime[i]:

                a.append(i)

   return a

def SearchNumber(uplimit,sdg):

   imax = 0

   # Declare list to be submitted to  eratosphen( )

   zn = []

   eratosphen(uplimit,zn)

   # Converting list "zn" into set 

   spisok = set(zn)

   for i in range(2,uplimit):

        # Look for highest prime number <= uplimit 

        # and summa of digits <= sdg

        if (i in spisok)  and sumDigits(i,10) <= sdg:

            if i > imax:

                 imax = i

   print("Maximum = ",imax)

def main():

   uplimit,sdg = 0,0

   uplimit = int(input("Input range of scan :"))

   sdg = int(input("Input sum of digits :"))

   SearchNumber(uplimit,sdg) 

if __name__ == "__main__":

   main()































Thursday, September 30, 2021

Attempt to test Web Cockpit Console on Fedora 35 Beta Server (VENV)

 First - Fedora 35 Beta Server deployed as L1 KVM Guest on F34 Bare metal Server . Nested virtualization enabled via virsh console on F34 Server. Complete KVM && Cockpit install performed on L1 F35 Server Guest per https://computingforgeeks.com/how-to-install-kvm-on-fedora/

 Second - Debian 11 L2 KVM Guest has been deployed via Web Cockpit Console on F35 Beta Server L1 Guest with no issues.

See http://lxer.com/module/forums/t/36755/

Debian 11 L2 Guest (UEFI mode installation) virtual drive has been intensionally configured with Debian Calamares Installer.











































Connection via ssh from F34 Bare metal Server instance to L2 Debian 11 Guest been deployed utilizing Cockpit Console set up on F35 Server L1 Guest















Obtain `df -Th` on Debian 11 L2 Guest















Display IP address of Debian 11 L2 Guest
















Tuesday, September 21, 2021

Sunday, September 19, 2021

Set up Virtual Box on top of Server F35 (pre release) via rpmfusion (VENV)

 First I've installed the most recent nightly build of Fedora 35 Server on Fedora 34 Bare metal KVM Virthost as Guest OS with "Fedora Workstation" desktop, like virtual machine seating on the Linux bridge been created via Web Cockpit Console. When done issued the following set of commands on F35 Guest

$ sudo dnf install  \

https://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-$(rpm -E %fedora).noarch.rpm 

$ sudo dnf  install \

https://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-$(rpm -E %fedora).noarch.rpm

$ sudo dnf install \

VirtualBox kernel-devel-$(uname -r) akmod-VirtualBox

$ sudo akmods

$ sudo systemctl restart vboxdrv 

$ lsmod  | grep -i vbox

Afterward performed a test install of Debian 11 as Oracle VBOX 6.1.26 virtual machine ( I did this installation in UEFI mode manually partitioning virtual drive with Calamares and nested kvm enabled )











































By the way, all Fedora 34 Bare metal Servers ( KVM VirtHosts ) are currently running under 5.14.5 fedora kernel. One Intel-based i4790, 2xBX500 SSD, another Ryzen 7/3700 based with WD500 NVMe SSD.

https://koji.fedoraproject.org/koji/packageinfo?packageID=8

Quick Enable

$ sudo dnf copr enable jforbes/kernel-stabilization



Wednesday, September 15, 2021

Python (numpy && matplotlib) solving one relatively difficult math problem

The problem itself is to find all solutions of the following equation on real axis

cos (cos (cos (cos (x)))) - sin (sin (sin (sin (x)))) = 0

We intend to prove that equation above doesn't have any solution via creating a python script importing NumPy and Matplotlib. Just recall that the function understudy has a period of 2*pi .

[boris@fedora34server ~]$ cat  yandexPlot5.py

import numpy as np # import numpy package

import matplotlib.pyplot as plt # import matplotlib.pyplot package

x = np.arange(0,9 * np.pi, 0.1) # create x array of angels from range 0 to 9*3.14

y = np.cos(np.cos(np.cos(np.cos(x)))) -  np.sin(np.sin(np.sin(np.sin(x)))) 

plt.plot(x, y) # plot grah 

plt.title(" Graphical Representation of sine function")

plt.xlabel("x axis ")

plt.ylabel("y axis ")

plt.show() # show plotted graph















Thus mentioned equation doesn't have any solution. Moreover increasing number of nested sines and cosines produces similar plotted graphs

































Wednesday, September 8, 2021

Deployment KVM Guests via Cockpit Console on Manjaro KDE 21.1.2

I am posting this Howto due to currently available instructions on the Internet don't focus the user's attention on installation packages "ovmf" and "virt-viewer" on the most recent release Manjaro KDE .  Skipping mentioned packages causes issues with Remote Viewer ( Cockpit Console ) and the ability to deploy KVM guests in UEFI mode

Install KVM and Cockpit Console

$ sudo pacman -S virt-manager qemu vde2 ebtables dnsmasq \
  bridge-utils openbsd-netcat virt-viewer ovmf
$ sudo systemctl enable --now libvirtd.service

$ sudo vi /etc/libvirt/libvirtd.conf
Uncomment
#unix_sock_group = "libvirt"
#unix_sock_ro_perms = "0777"
#unix_sock_ro_perms = "0777"

$ sudo systemctl restart libvirtd.service

Cockpit console setup
$ sudo pacman -S cockpit
$ sudo systemctl enable --now cockpit.socket

Tuning firewall
$ sudo apt install firewalld
$ sudo systemctl enable --now firewalld
$ sudo firewall-cmd --add-service=cockpit --permanent
$ sudo firewall-cmd --reload









































Thursday, August 19, 2021

Set up KVM && Cockpit WEB Console on Debian Bullseye (11)

UPDATE 08/31/21
In case setting up Debian 11 with KDE you would have manually select "Remote Viewer'  from drop-down menu inside Cockpit Console. It won't be detected automatically like in Debian 11 with Gnome desktop




















END UPDATE
UPDATE as of 08/29/21
After kernel upgrade up to 5.13-trunk-amd64 . KVM Hypervisor seems to keep working stable. See How to Install Backports & Experimental Repository on Debian 11
END UPDATE

The presence of Web Cockpit Console is a nice way to manage KVM guest's deployment via clicking the button "Remote Virt-viewer"  built into Cockpit Web Console. Bridge attached to external network interface was also created pretty smoothly utilizing network management section inside Web Console. However, it doesn't look to me as a reason to deprecate virt-manager . Virt-manager still provides a way to edit guest definitions which can be very handy and  sometimes seems to be more flexible versus Cockpit Web console in my very personal opinion.

 Install KVM

 $ sudo apt install qemu-kvm  libvirt-daemon bridge-utils \
         virtinst libvirt-daemon-system -y
 $ sudo apt install  libguestfs-tools libosinfo-bin \
       qemu-system virt-manager -y
 $  sudo modprobe vhost_net 
 $  lsmod | grep vhost
 $  echo vhost_net | sudo tee -a /etc/modules
 $  sudo usermod -a -G libvirt  $(whoami)
 $  sudo reboot

 Install Web Cockpit Console
 $  sudo apt install install cockpit cockpit-machines
 $  sudo apt install cockpit cockpit-machines
 $  sudo systemctl start cockpit.socket
 $  sudo systemctl enable cockpit.socket

 Tuning firewall
 $  sudo apt install firewalld
 $  sudo firewall-cmd --add-service=cockpit --permanent
 $  sudo firewall-cmd --reload