Python libraries

From wikinotes
Revision as of 05:17, 3 February 2023 by Will (talk | contribs) (→‎Misc)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

Outside of the main python language features (which are unlikely to change), this documents individual python modules (internal and external). Some of these are referenced outside of this document (those I find most useful), but consider this to be the master list.

Authentication

python pyjwt
python htpasswd
python crypt
python bcrypt
python passlib
python keyring

Data Analytics

python pandas
python scipy
python numpy

Databases

sqlalchemy Interface that abstracts different database types. Also provides an ORM.
mysql-python python2x bindings to the C MySQL API
pymysql pure python mysql interface for python2/3
psycopg2 postgres database interface
couchdb-python python interface for couchdb
python-pouchdb python bindings to interact with pouchdb(javascript)
sqlite3 builtin support for sqlite
python couchbase lite c python client for couchbase-lite-c
python redis python client for redis

Data Serialization

python csv builtin csv parsing
configobj nested .ini files
configparser ini file reader/writer
json json module
yaml yaml parsing modules
etree builtin XML parser
feedparser reading RSS feeds

Data Visualization

python matplotlib bargraphs, scatterplots, etc
python termplotlib aims to be equiv to matplotlib, but rendered in ascii
python manim create math animations (used by 3blue 1brown)

Debuggers

python pdb the official builtin debugger
python ipdb the ipython debugger, built overtop of pdb with colours, tab-completion, and enhanced featureset
python trepan Formerly pydbgr, pygdb, extends pdb to include features from gdb.
python pudb TUI console deugger, that shows your code while debugging.
gdb#python debug frameworks that were compiled in C/Cpp (qt, etc).

File Management

GitPython python interface for git
tarfile tarfile
zipfile zipfile

Freezing/Compiling

cython compile python to C/C++
cx_freeze freeze python into executables
pyoxidizer embed python in rust executable (single file executable, statically linked)
py2exe py2exe allows you to compile python executables for windows
python nuitka native executable compiler for python

Linting

pylint automated checks for your codebase's consistency with the various PEPs, and established bad practices.

Media

python wave read/write wav files
python pyaudio (external) cross platform audio recording/playback
python samplerate (external) algorithms to modify wavfile samplerates (numpy arrays)
python wavio (external) wavfile to numpy conversions and back
python resampy (external) manipulate wavfile samplerates in numpy arrays

Misc

openpyxl Interface with Excel Speadsheets (.xlsx)
pypdf2 PDF authoring/manipulation
pypdf4 PDF authoring/manipulation
pdfrw PDF authoring/manipulation
pdfreader read/extract info from pdfs
icalendar icalendar .ics read/write tool
hashlib md5, sha, ... hashing library
lockfile entirely cross-platform lock/pid file mechanism
datetime native python object for managing times, timezones
pytz timezone info for python
ctypes interfaces with C libraries.
interface java-style interface implementation in python.
docutils convert between text formats (html, man, tex, xml, rst, ...)
python orgparse parse org mode files, clock times, etc.

Operating Systems

General

pyxdg XDG-compatible config,data,cache directory finding
python-daemon facilitates daemon start/stop and pidfiles
watchdog observes filesystem events, notifies python callables.
pyautogui cross-platform keystroke, mouse, and selection controls

Unix

pydbus Python interface for dbus
python-xlib xlib bindings for python (Xorg)
python xcffib xcb bindings for python (Xorg)

Windows

python pyhook grab user-keystrokes
python pywin32 the win32 com interface, exposed to python
python msilib python bindings for windows installer API

Packaging

setuptools creating/installing software distrobutions (and uploading to PyPi).
virtualenv safely manage a set of python requirements within an environment

Performance Testing

cProfile python's builtin profiling software. The basis for most other profiling tools.
pprofile line-profiling.
line_profiler after identifying a problem area, line_profiler allows you to test your code line-by-line.
runSnakeRun My favourite profiling tool. This has saved my hide an inummerable amount of times.
python snakeviz Decent profiling tool, supports python3
vprof Hierarchical view of your code, and how long it took to run. Provides heat maps for how long each line in your code took to run.
timeit watches time it take to run code
gitviewer This was mentioned in one of Alex Gaynor's(Pypy) talks about the speed of python . I haven't been able to find it online. It is supposed to show each line of code, it's speed, it's call to the Python VM, and it's sysCall. It was supposed to be a tool for helping you identify good/bad code practices.
pympler memory profiler for python
objgraph find memory leaks by filtering/identifying instances, and examing their parent graph using graphviz
resource bulitin library to examine/manage progam's python resource usage (memory etc)

Progam Extension

taskw python taskwarrior API

Proprietary APIs

python boto3 amazon S3 client

Refactoring

pycharm A python IDE. honestly, best refactoring experience I've seen for python.
rope A refactoring library for python

Sockets/RPC

netifaces cross-platform python API for network interfaces/connections
flask RPC/Rest API tools (requires a webserver)
json-rpc (Remote Procedure Call) using JSON (works over web sockets)

Static Types

python typing
python mypy

Speech To Text

python SpeechRecognition abstracts several speech-to-text backends, listens on mic
mozilla deepspeech one speech-to-text backend, with python bindings
python pocketsphinx python bindings for CMUsphinx speech-to-text
python pyjsgf helper library for working with JSGF files (defines command grammar)

Templating

python jinja2 templating language
python mako templating language

User Interfaces

GUI

python qt
python wx

CLI

argparse CLI argument parser
docopt CLI argument parser
prompt_toolkit syntax-highlighting, tab-completion, repl... so easy it's sexy
pygments syntax-highlighting, and printing colours to console

TUI

python curses lowlevel graphics rendering library for the terminal
python notcurses python bindings for notcurses C library
python urwid modern tex-interfaces using widgets (built on top of curses)
python reprint simple way of reprinting multiple lines on a console

Unit Testing

pytest (++) A-Mazing test-runner AND unittest framework.
mock (++) test/fake function results for unittests
pytest-cov test code-coverage while testing in pytest
coverage
unittest A unittesting framework based off of the popular XUnit library.
unittest2 ??
pyfakefs mock out the filesystem in memory
pyramid ??
nose customized unittest runner (unmaintained)
nose2 followup to nose
tox run unittests in multiple versions of python at once (using virtualenvs)
dox similar to tox, but runs multiple versions of python in multiple linux distrobutions by using docker on the back-end.

Web

python adblock python wrapper for brave's adblocker
python requests less verbose http requests
python http tools for http servers, requests, cookies
python graphene build graphql apis
python flask dynamic web interface, REST api, etc.
python django website creation framework
python gunicorn WSGI Servers
python uvicorn ASGI Servers