i like to work with fastapi. “FastAPI is a modern, fast (high-performance), web framework for building APIs with Python based on standard Python type hints.”. i also know that scripting languages like python are ways slower than compiled languages like c, c++, rust, …
why not build a little “hello world” api, running it on localhost, and then do a benchmark …
let’s start with rust.
src/main.rs
Long time ago, there were huge discussions about Python 2.7 or Python 3.xx. Fortunately, these times are gone and we’ve all gotten over the hurdle to Python 3. But are you on 3.6, 3.7, 3.8 ? or even 3.11 or 3.12 ? That’s the current Version you should use for your daily Projects ?
A good indicator is this Website: https://devguide.python.org/versions/
It’s also recommended to check what our Operating System is installing by default, or what you can get from their Package Repository.
… will be released next week (23. Oct 2023). Why not have a look at the upcomming OS and prepare a VM for Software Development ?
grab a fresh VM and Install OpenBSD 7.4
os version
puffy74# sysctl kern.version
kern.version=OpenBSD 7.4 (GENERIC.MP) #1396: Sun Oct 8 09:20:40 MDT 2023
deraadt@amd64.openbsd.org:/usr/src/sys/arch/amd64/compile/GENERIC.MP
empty vm
puffy74# pkg_info
quirks-6.159 exceptions to pkg_add rules and cache
puffy74# pkg_add go rust python3
quirks-6.159 signed on 2023-10-07T10:09:24Z
go-1.21.1: ok
rust-1.72.1:nghttp2-1.56.0: ok
rust-1.72.1:nghttp3-0.15.0: ok
rust-1.72.1:ngtcp2-0.19.1: ok
rust-1.72.1:curl-8.3.0p0: ok
rust-1.72.1:libssh2-1.11.0: ok
rust-1.72.1: ok
python3-3.10p2:xz-5.4.4: ok
python3-3.10p2:sqlite3-3.42.0: ok
python3-3.10p2:libiconv-1.17: ok
python3-3.10p2:gettext-runtime-0.22.2: ok
python3-3.10p2:libffi-3.4.4: ok
python3-3.10p2:bzip2-1.0.8p0: ok
python3-3.10p2:python-3.10.13: ok
python3-3.10p2: ok
show packages
cd /some/path/you/want
poetry new mypackage
poetry add requests
cat << 'EOF' > mypackage/__init__.py
print("importing", __name__)
EOF
cat << 'EOF' > mypackage/main.py
print("importing", __name__)
def test1():
print("test1")
def test2(name: str):
print("hello", name)
def test3(name: str, age:int):
print(f"Hello {name} at age {age}")
if __name__ == "__main__":
print("This is a Library or Package. You should import it into your Code and not run it directly ...")
EOF
poetry build
(mypackage-py3.11) stoege@host:~/git/demo/mypackage> tree
.
├── README.md
├── dist
│ ├── mypackage-0.1.0-py3-none-any.whl
│ └── mypackage-0.1.0.tar.gz
├── mypackage
│ ├── __init__.py
│ └── main.py
├── poetry.lock
├── pyproject.toml
└── tests
└── __init__.py
4 directories, 8 files
you have a package called “mypackage-0.1.0” created. As ’tar.gz and ‘.whl’ File
Small Example how to Store Data in JSON, and Query them afterwards like a NOSQL DB. Have a look at TinyDB if you wanna see more.
from tinydb import TinyDB, Query
from pprint import pprint
# Create or load a database file
db = TinyDB('db.json')
# insert some sample data
def insert():
# Insert data
db.insert({'name': 'John', 'age': 30})
db.insert({'name': 'Alice', 'age': 25, 'hobbies': 'sleep'})
db.insert({'name': 'Max', 'age': 20, 'hobbies': ['sleep', 'play', 'eat']})
# show all entries
def show_all():
all_records = db.all()
pprint(all_records)
# entries with hobbies
def show_entries_with_hobbies():
User = Query()
result = db.search(User.hobbies.exists())
pprint(result)
# entries without hobbies
def show_entries_without_hobbies():
User = Query()
result = db.search(~User.hobbies.exists())
pprint(result)
# show entries with hobbies and older than 22 years
def show_entries_with_hobbies_and_older_than_22():
User = Query()
result = db.search((User.hobbies.exists()) & (User.age > 22))
pprint(result)
if __name__ == "__main__":
# Add
insert()
# show
print("\n-- ALL --")
show_all()
print("\n-- with Hobbies --")
show_entries_with_hobbies()
print("\n-- without Hobbies --")
show_entries_without_hobbies()
print("\n-- with Hobbies and older than 22 --")
show_entries_with_hobbies_and_older_than_22()
you need to install tinydb. use a virtual env like .venv, poetry or whatever you like
gave a try with a FastAPI Template, https://github.com/rochacbruno/fastapi-project-template.git
clone the repo
git clone https://github.com/rochacbruno/fastapi-project-template.git gugus1234
cd gugus1234
i’d like to have poetry as virtual env manager
make switch-to-poetry
had to rename some string in pyproject and different files …
mv project_name gugug1234
gsed -i 's/a-flask-test/gugus1234/' pyproject.toml
gsed -i 's/project_name/gugus1234/' pyproject.toml
gsed -i 's/project_name/gugus1234/g' gugus1234/cli.py gugus1234/app.py gugus1234/config.py gugus1234/security.py
poetry shell
poetry lock
poetry install
let’s create admin user
Let’s build a small API Endpoint with FastAPI and protect it with SimpleSecurity.
API key based security package for FastAPI, focused on simplicity of use:
and show the Directory Structure
cat <<'EOF'> ping.py
import argparse
from ping3 import ping, verbose_ping
def do_ping(host: str, timeout: int = 3, size: int = 1500, output: str = "json"):
# output: json|txt
# '21.54 ms'
if size > 1500:
size = 1500
result = (
str(
round(
ping(dest_addr=host, timeout=timeout, size=size, unit="ms"),
2,
)
)
+ " ms"
)
if output.lower() == "json":
return {"host": host, "timeout": timeout, "size": size, "result": result}
if output.lower() == "txt":
return result
else:
return f"output format '{output} unknown! use 'json|txt'"
def do_multiple_ping(host: str, count: int = 3, interval: float = 0):
# ping 'www.stoege.net' ... 23ms
# ping 'www.stoege.net' ... 24ms
# ping 'www.stoege.net' ... 20ms
verbose_ping(
dest_addr=host,
count=count,
interval=interval,
)
def main():
# Create the argument parser
parser = argparse.ArgumentParser(description="Ping a domain or IP address.")
# Add the host argument
parser.add_argument(
"host",
metavar="HOST",
type=str,
nargs="?",
default="www.stoege.net",
help="the domain or IP address to ping",
)
# Parse the command-line arguments
args = parser.parse_args()
# Call the ping function
output = do_ping(host=args.host, output="json")
# Print the ping output
print(f"\n{output}\n")
# Call the ping function. No return Value !
do_multiple_ping(host=args.host, count=10, interval=0.1)
if __name__ == "__main__":
main()
EOF
poetry, venv, whatever you like
let’s tune the default logger a bit so he write nice and colored messages.
a little config File …
cat <<'EOF'> config.py
LOGGER_MAX_FILE_LENGTH = 10
EOF
the logger code in the ‘src’ Folder
mkdir src
cat <<'EOF'> src/logger.py
import logging
import datetime
import sys
from config import *
if isinstance(LOGGER_MAX_FILE_LENGTH, int):
LOGGER_MAX_FILE_LENGTH = str(LOGGER_MAX_FILE_LENGTH)
def get_now() -> str:
#
# choose your format
#
current_time = datetime.datetime.now()
# 2023-07-16 22:16:15.958
formatted_time_1 = current_time.strftime("%Y-%m-%d %H:%M:%S.%f")[:-3]
# 22:16:54.471
formatted_time_2 = current_time.strftime("%H:%M:%S.%f")[:-3]
# 22:17:21
formatted_time_3 = current_time.strftime("%H:%M:%S")
return formatted_time_2
class ExitOnCriticalHandler(logging.Handler):
def emit(self, record):
# Unset Color
COLOR = RESET = "\033[0m"
# Debug -> Black
if record.levelno == logging.DEBUG:
COLOR = "\033[0m"
# Info -> Green
elif record.levelno == logging.INFO:
COLOR = "\033[92m"
# Warn -> Blue
elif record.levelno == logging.WARNING:
COLOR = "\033[94m"
# Error -> Orange
elif record.levelno == logging.ERROR:
COLOR = "\033[38;5;208m"
# Critical -> Red
elif record.levelno >= logging.CRITICAL:
COLOR = "\033[91m"
# Custom Line
print(
COLOR
+ "{:} {:} {:04} {:{w}} {:}".format(
get_now(),
record.levelname,
record.lineno,
record.filename,
record.msg,
w=LOGGER_MAX_FILE_LENGTH,
)
+ RESET
)
# Exit on Critical
if record.levelno >= logging.CRITICAL:
logging.shutdown()
print("\033[91m" + "GOT A CRITICAL -> EXIT HERE!" + "\033[0m")
sys.exit()
# Init Logger
logger = logging.getLogger()
logger.setLevel(logging.INFO)
# Create and add the custom handler
exit_handler = ExitOnCriticalHandler()
logger.addHandler(exit_handler)
# Set Log Level
def set_log_level(level: str):
# Parse/ Set LogLevel
if level.lower() in ["d", "debug"]:
logger.setLevel(logging.DEBUG)
elif level.lower() in ["i", "info"]:
logger.setLevel(logging.INFO)
elif level.lower() in ["w", "warning"]:
logger.setLevel(logging.WARNING)
elif level.lower() in ["e", "error"]:
logger.setLevel(logging.ERROR)
elif level.lower() in ["c", "critical"]:
logger.setLevel(logging.CRITICAL)
# Custom level name mappings, Debug -> D
logging.addLevelName(logging.DEBUG, "D")
logging.addLevelName(logging.INFO, "I")
logging.addLevelName(logging.WARNING, "W")
logging.addLevelName(logging.ERROR, "E")
logging.addLevelName(logging.CRITICAL, "C")
# Functions to Call
def ldebug(msg: str):
logger.debug(msg)
def linfo(msg: str):
logger.info(msg)
def lwarning(msg: str):
logger.warning(msg)
def lerror(msg: str):
logger.error(msg)
def lcritical(msg: str):
logger.critical(msg)
EOF
the Main File with Argparse to set the Logging Level on Startup
cat << EOF > main.py
a = "top"
b = "secret"
print("This is", a, b)
EOF
python3 main.py
This is top secret
doas poetry self update
poetry self update
or
pip install poetry -U
poetry init
poetry add pyinstaller
poetry run pyinstaller main.py --onefile
ls -la dist/
file dist/main
ls -la dist/
4735533 Jun 26 22:17 main
```.sh
file dist/main
file dist/main
dist/main: ELF 64-bit LSB shared object, x86-64, version 1
./dist/main
./dist/main
Hello World! geheim
strings dist/main |sort
strings dist/main |sort
...
unsetenv
vfprintf
vsnprintf
w3M9
waitpid
wcsncpy
|$8H
~$E1
strings dist/main |grep -E "top|secret"
strings dist/main |grep -E "top|secret"
# -> nothing found
objdump -x dist/main
objdump -x dist/main
dist/main: file format elf64-x86-64
dist/main
architecture: i386:x86-64, flags 0x00000150:
HAS_SYMS, DYNAMIC, D_PAGED
start address 0x0000000000005ed0
Program Header:
PHDR off 0x0000000000000040 vaddr 0x0000000000000040 paddr 0x0000000000000040 align 2**3
filesz 0x00000000000002a0 memsz 0x00000000000002a0 flags r--
INTERP off 0x00000000000002e0 vaddr 0x00000000000002e0 paddr 0x00000000000002e0 align 2**0
filesz 0x0000000000000013 memsz 0x0000000000000013 flags r--
LOAD off 0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**12
filesz 0x0000000000004ecc memsz 0x0000000000004ecc flags r--
LOAD off 0x0000000000004ed0 vaddr 0x0000000000005ed0 paddr 0x0000000000005ed0 align 2**12
filesz 0x0000000000007700 memsz 0x0000000000007700 flags --x
LOAD off 0x000000000000c5d0 vaddr 0x000000000000e5d0 paddr 0x000000000000e5d0 align 2**12
filesz 0x0000000000000918 memsz 0x0000000000000918 flags rw-
LOAD off 0x000000000000cee8 vaddr 0x000000000000fee8 paddr 0x000000000000fee8 align 2**12
filesz 0x0000000000000004 memsz 0x000000000000602c flags rw-
DYNAMIC off 0x000000000000c900 vaddr 0x000000000000e900 paddr 0x000000000000e900 align 2**3
filesz 0x0000000000000140 memsz 0x0000000000000140 flags rw-
RELRO off 0x000000000000c5d0 vaddr 0x000000000000e5d0 paddr 0x000000000000e5d0 align 2**0
filesz 0x0000000000000918 memsz 0x0000000000000a30 flags r--
EH_FRAME off 0x0000000000003c1c vaddr 0x0000000000003c1c paddr 0x0000000000003c1c align 2**2
filesz 0x000000000000030c memsz 0x000000000000030c flags r--
OPENBSD_RANDOMIZE off 0x000000000000c5d0 vaddr 0x000000000000e5d0 paddr 0x000000000000e5d0 align 2**3
filesz 0x0000000000000308 memsz 0x0000000000000308 flags rw-
STACK off 0x0000000000000000 vaddr 0x0000000000000000 paddr 0x0000000000000000 align 2**0
filesz 0x0000000000000000 memsz 0x0000000000000000 flags rw-
NOTE off 0x00000000000002f4 vaddr 0x00000000000002f4 paddr 0x00000000000002f4 align 2**2
filesz 0x0000000000000018 memsz 0x0000000000000018 flags r--
Dynamic Section:
NEEDED libm.so.10.1
NEEDED libz.so.7.0
NEEDED libc.so.97.0
FLAGS_1 0x8000000
DEBUG 0x0
RELA 0xf50
RELASZ 0x6d8
RELAENT 0x18
RELACOUNT 0x46
JMPREL 0x1628
PLTRELSZ 0x6d8
PLTGOT 0xec88
PLTREL 0x7
SYMTAB 0x310
SYMENT 0x18
STRTAB 0xcd8
STRSZ 0x272
GNU_HASH 0xa48
HASH 0xa68
Sections:
Idx Name Size VMA LMA File off Algn
0 .interp 00000013 00000000000002e0 00000000000002e0 000002e0 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
1 .note.openbsd.ident 00000018 00000000000002f4 00000000000002f4 000002f4 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
2 .dynsym 00000738 0000000000000310 0000000000000310 00000310 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
3 .gnu.hash 00000020 0000000000000a48 0000000000000a48 00000a48 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .hash 00000270 0000000000000a68 0000000000000a68 00000a68 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
5 .dynstr 00000272 0000000000000cd8 0000000000000cd8 00000cd8 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
6 .rela.dyn 000006d8 0000000000000f50 0000000000000f50 00000f50 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
7 .rela.plt 000006d8 0000000000001628 0000000000001628 00001628 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
8 .rodata 00001f1c 0000000000001d00 0000000000001d00 00001d00 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
9 .eh_frame_hdr 0000030c 0000000000003c1c 0000000000003c1c 00003c1c 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
10 .eh_frame 00000fa4 0000000000003f28 0000000000003f28 00003f28 2**3
CONTENTS, ALLOC, LOAD, READONLY, DATA
11 .text 00006d90 0000000000005ed0 0000000000005ed0 00004ed0 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
12 .init 0000000e 000000000000cc60 000000000000cc60 0000bc60 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
13 .fini 0000000e 000000000000cc70 000000000000cc70 0000bc70 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
14 .plt 00000950 000000000000cc80 000000000000cc80 0000bc80 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
15 .openbsd.randomdata 00000308 000000000000e5d0 000000000000e5d0 0000c5d0 2**3
CONTENTS, ALLOC, LOAD, DATA
16 .jcr 00000008 000000000000e8d8 000000000000e8d8 0000c8d8 2**3
CONTENTS, ALLOC, LOAD, DATA
17 .ctors 00000010 000000000000e8e0 000000000000e8e0 0000c8e0 2**3
CONTENTS, ALLOC, LOAD, DATA
18 .dtors 00000010 000000000000e8f0 000000000000e8f0 0000c8f0 2**3
CONTENTS, ALLOC, LOAD, DATA
19 .dynamic 00000140 000000000000e900 000000000000e900 0000c900 2**3
CONTENTS, ALLOC, LOAD, DATA
20 .got 00000248 000000000000ea40 000000000000ea40 0000ca40 2**3
CONTENTS, ALLOC, LOAD, DATA
21 .got.plt 00000260 000000000000ec88 000000000000ec88 0000cc88 2**3
CONTENTS, ALLOC, LOAD, DATA
22 .data 00000004 000000000000fee8 000000000000fee8 0000cee8 2**2
CONTENTS, ALLOC, LOAD, DATA
23 .bss 00006024 000000000000fef0 000000000000fef0 0000ceec 2**4
ALLOC
SYMBOL TABLE:
no symbols
objdump -d dist/main
objdump -d dist/main |head -100
dist/main: file format elf64-x86-64
Disassembly of section .text:
0000000000005ed0 <.text>:
5ed0: 48 89 d1 mov %rdx,%rcx
5ed3: 48 8b 3c 24 mov (%rsp),%rdi
5ed7: 48 8d 54 fc 10 lea 0x10(%rsp,%rdi,8),%rdx
5edc: 48 8d 74 24 08 lea 0x8(%rsp),%rsi
5ee1: 48 83 ec 08 sub $0x8,%rsp
5ee5: 48 83 e4 f0 and $0xfffffffffffffff0,%rsp
5ee9: 48 83 c4 08 add $0x8,%rsp
5eed: eb 01 jmp 5ef0 <_csu_finish@plt-0x6da0>
5eef: cc int3
5ef0: 55 push %rbp
5ef1: 48 89 e5 mov %rsp,%rbp
5ef4: 41 57 push %r15
5ef6: 41 56 push %r14
5ef8: 41 55 push %r13
5efa: 41 54 push %r12
5efc: 53 push %rbx
5efd: 50 push %rax
5efe: 49 89 cd mov %rcx,%r13
5f01: 49 89 d4 mov %rdx,%r12
5f04: 49 89 f6 mov %rsi,%r14
5f07: 41 89 ff mov %edi,%r15d
5f0a: 48 89 f7 mov %rsi,%rdi
5f0d: 48 89 d6 mov %rdx,%rsi
5f10: 48 8b d1 mov %rcx,%rdx
5f13: e8 98 6d 00 00 callq ccb0 <_Jv_RegisterClasses@plt>
5f18: 48 8b d8 mov %rax,%rbx
5f1b: 4d 85 ed test %r13,%r13
5f1e: 0f 85 db 00 00 00 jne 5fff <_csu_finish@plt-0x6c91>
5f24: 48 89 5d d0 mov %rbx,0xffffffffffffffd0(%rbp)
5f28: 48 8d 1d a1 ff ff ff lea -95(%rip),%rbx # 5ed0 <_csu_finish@plt-0x6dc0>
5f2f: 48 8d 0d 9a ff ff ff lea -102(%rip),%rcx # 5ed0 <_csu_finish@plt-0x6dc0>
5f36: 48 29 d9 sub %rbx,%rcx
5f39: 48 8d 41 07 lea 0x7(%rcx),%rax
5f3d: 48 85 c9 test %rcx,%rcx
5f40: 48 8b d1 mov %rcx,%rdx
5f43: 48 0f 48 d0 cmovs %rax,%rdx
5f47: 48 c1 fa 03 sar $0x3,%rdx
5f4b: 74 46 je 5f93 <_csu_finish@plt-0x6cfd>
5f4d: 48 85 c9 test %rcx,%rcx
5f50: 48 0f 49 c1 cmovns %rcx,%rax
5f54: 48 c1 f8 03 sar $0x3,%rax
5f58: 48 83 f8 02 cmp $0x2,%rax
5f5c: 41 bd 01 00 00 00 mov $0x1,%r13d
5f62: 4c 0f 43 e8 cmovae %rax,%r13
5f66: eb 08 jmp 5f70 <_csu_finish@plt-0x6d20>
5f68: cc int3
5f69: cc int3
5f6a: cc int3
5f6b: cc int3
5f6c: cc int3
5f6d: cc int3
5f6e: cc int3
5f6f: cc int3
5f70: 4c 8b 1b mov (%rbx),%r11
5f73: 44 89 ff mov %r15d,%edi
5f76: 4c 89 f6 mov %r14,%rsi
5f79: 4c 89 e2 mov %r12,%rdx
5f7c: 31 c9 xor %ecx,%ecx
5f7e: e8 9d 00 00 00 callq 6020 <_csu_finish@plt-0x6c70>
5f83: 48 87 d8 xchg %rbx,%rax
5f86: 48 83 c0 08 add $0x8,%rax
5f8a: 48 87 d8 xchg %rbx,%rax
5f8d: 49 83 c5 ff add $0xffffffffffffffff,%r13
5f91: 75 dd jne 5f70 <_csu_finish@plt-0x6d20>
5f93: 4c 8d 2d 36 ff ff ff lea -202(%rip),%r13 # 5ed0 <_csu_finish@plt-0x6dc0>
5f9a: 48 8d 0d 2f ff ff ff lea -209(%rip),%rcx # 5ed0 <_csu_finish@plt-0x6dc0>
5fa1: 4c 29 e9 sub %r13,%rcx
5fa4: 48 8d 41 07 lea 0x7(%rcx),%rax
5fa8: 48 85 c9 test %rcx,%rcx
5fab: 48 8b d1 mov %rcx,%rdx
5fae: 48 0f 48 d0 cmovs %rax,%rdx
5fb2: 48 c1 fa 03 sar $0x3,%rdx
5fb6: 74 3c je 5ff4 <_csu_finish@plt-0x6c9c>
5fb8: 48 85 c9 test %rcx,%rcx
5fbb: 48 0f 49 c1 cmovns %rcx,%rax
5fbf: 48 c1 f8 03 sar $0x3,%rax
5fc3: 48 83 f8 02 cmp $0x2,%rax
5fc7: bb 01 00 00 00 mov $0x1,%ebx
5fcc: 48 0f 43 d8 cmovae %rax,%rbx
5fd0: 4d 8b 5d 00 mov 0x0(%r13),%r11
5fd4: 44 89 ff mov %r15d,%edi
5fd7: 4c 89 f6 mov %r14,%rsi
5fda: 4c 89 e2 mov %r12,%rdx
5fdd: 31 c9 xor %ecx,%ecx
5fdf: e8 3c 00 00 00 callq 6020 <_csu_finish@plt-0x6c70>
5fe4: 49 83 c5 08 add $0x8,%r13
5fe8: 48 87 d8 xchg %rbx,%rax
5feb: 48 83 c0 ff add $0xffffffffffffffff,%rax
5fef: 48 87 d8 xchg %rbx,%rax
5ff2: 75 dc jne 5fd0 <_csu_finish@plt-0x6cc0>
5ff4: c6 05 45 9f 00 00 01 movb $0x1,40773(%rip) # ff40 <memcmp@plt+0x2e30>
5ffb: 48 8b 5d d0 mov 0xffffffffffffffd0(%rbp),%rbx
5fff: e8 5c 6c 00 00 callq cc60 <_csu_finish@plt-0x30>
6004: 48 8b 13 mov (%rbx),%rdx
...
-> at least the variables are not obviously found in the binary. but don’t want to know what ghidra says about it :(