Full Training Programs
Free Online Training Events
Free Technical Resources
COVID-19 Update: View Doulos COVID News Updates »
This course is available Live Online worldwide: View the Live Online full course description »
In-person scheduled training is now available on a limited basis in some areas.
Please view the course calendar or contact Doulos for team training options.
Python is a rapid prototyping language. With a small amount of code (say a few dozen or hundreds of lines), comes the ability to automate or simplify daily, tedious recurring tasks and save time - getting a quick return on investment!
If the tool is useful, what started as "a little-project" suddenly becomes used by more and more people; several developers start working on enhancing the tool and new functionalities keep creeping in organically…
This is a typical journey. You build Python tools to simplify your daily work, only to discover later that it has become a "software product", and your core business may not even be software products to begin with!
So, you're now facing a whole new set of challenges:
This course offers a blueprint to build “future-proof” software Python products, regardless of whether you're starting afresh or from an existing code base.
This training is delivered as a 4-day public face-to-face training or 5 sessions of live online training.
Workshops comprise approximately 50% of class time and are based around carefully designed hands-on exercises to reinforce learning.
Course overview video:
Expert Product Development with Python is a hands-on programming course aimed at software, hardware, support engineers, or anyone experienced in using Python code, who wants to create or turn an existing Python code base into a software product.
Typical applications include: command line and tools flows, for hardware verification, for software tests, for data science and machine learning, or for running on embedded devices.
This course does not assume any prior experience in productizing software. Of course, it won’t hurt if you do.
To maximize the training value, prior attendance of the Doulos course Essential Python (or equivalent experience) is required.
It is assumed that the participants are, or have been, programming with Python on a regular basis and have the relevant experience with the topics taught in Essential Python.
Doulos training materials are renowned for being the most comprehensive and user friendly available. Their style, content and coverage is unique, and has made them sought after resources in their own right. The materials include:
Introducing Product Development with Python
From organic grown scripts to product • Training Goal • Course Roadmap • Modules Overview
An advanced tour of Python's essential features
Name Binding • Compiled or Interpreted? • Global vs Local • LEGB Scoping Rules • Argument Passing • Mutability • Function Arguments • Special Parameters • Object in Python • First Class Objects • Decorator • Lambda, Iterable and Comprehension • F-strings • Virtual Environment
Efficient debugging tools and techniques in Python
Builtins Introspection • inspect • inspect stack frame • debugging playground • starting pdb • pdb commands • interactive session • Breakpoints • Inspect and modify • pdb Cheat Sheet • Switching debugger • Visual debugger • The Scientific Method • Going Further
Advanced Pythonic object-oriented programming techniques
Class 101 • Class methods • Alternative Constructors • Static Methods •Virtual methods • Private Name Mangling • Instance variables vs getter/setter • property • property setter and deleter • Interface and ABC • ABC in Python • ABC and collections • Multiple Inheritance • MRO • Multiple Inheritance and Interfaces • Mixins • __slots__• Bonus Annex
A deeper understanding of modules and packages and how to use them
import • import under the hood • module caching • from…import • import * • __name__ • packages • import from Package • Running modules from the command line • logical namespace • practical considerations.
Ways to document and structure code for peers and code users
Documentation strings • documenting API • type hints • mypy • code formatting • black • isort
How to create a distribution package
introduction • Really Simple Script (rss) • toward productization • Python Project Layout • pyproject.toml • Configuring Metadata • Distribution Package Metadata • classifiers • package dependencies • Controlling Package Contents • Prod/Dev Environment • Configuring Black and Isort • Python Application • Testing your Distribution Package • Install for Source • Building • Going Further • Bonus Annex
Introducing pytest and common tools and techniques used for testing python products
Introduction • Testing Considerations • Testing Strategy • The Four-Phase Test pattern • Fake and Test Doubles • Doctest for e2e Testing • Doctest and docstring • Pytest Architecture • A Simple Test • A Failing Test • Test Discovery • Grouping Tests in a Class • Testing that an Exception is Raised • Skipped Tests and Expected Failures • Parametrization • Test Summary Report • Dropping into pdb • Test Fixtures • User-Defined Test Fixtures • Mocking • Monkeypatch Fixtures • Monkeypatch and Mocking Fixtures • pytest Good Practices
The Python logging module
Logging: An overview • Loggers • Log level • Handlers • Formatters • Loggers hierarchy • Filters • Logging Workflow • Practical Consideration • Config by Coding • Logging on an Embedded System • Additional Resources • Bonus Annex
How to create a sound command line interface in Python
Introduction • CLI Design Guidelines • Exiting Python Programs •Handling Signal • Custom Signal Handler • Dealing with Ctrl-C • Parsing Argument • Positional Arguments • Usage and Help • Option Arguments • Options With and Without Values • Optional Values • nargs • Description, Epilog, Help • Prefix, Choices, Required • Click • Command Examples • From argparse to click • Click Path and File • Command Group • Invoking Sub-commands• Improving User Experience • Tqdm Progress Bar • Rich
Common tools and techniques to assess and improve the quality of your python product
Static Analysis and Type Checkers • flake8 • pylint • Security Auditing • Practical Considerations • Integrating Static Analysis Tools • Type checkers • Code Coverage • Coverage Report • Coverage Area • Branch Coverage • Mutation Testing • mutmut Output • Analysing mutmut Report • Tips and Tricks
An overview of concurrency and available modules from the standard library
What is concurrency • CPU vs I/O bound tasks • Concurrent Programming Models • Concurrency in Python • Concurrency Modules • Modules Summary
Introducing the threading module to implement concurrency using OS-threads
Introduction • Thread Creation • Thread Creation Using Class • Thread and Program Termination • Joining Thread • Thread Local Data • Race Condition • Synchronizations Objects • Lock Object • Event Object • Condition Object (waiters) • Condition Object (notify) • Communicating between Threads • Working with the GIL • Bonus annex
Introducing the multiprocessing module to implement concurrency using OS processes
Introduction • Starting a New Process • Start Methods • Choosing a Start Method • Inheriting Resources • Join, Terminate and Exit Code • Inter-Process Communication • Pipe • Queues • Shared Memory • Manager • Process Pool • Task Offload Methods • Bonus Annex
Some of the high-level constructs provided by the concurrent.futures module
Introduction • Executor Object • Thread Executor Pool • Process Executor Pool • Future • Future and Deadlocks • Customizing Future Execution
Introducing the asyncio module to implement concurrency using Python coroutines
Introduction • Async IO architecture • async, await and coroutines • asyncio.run() Error • asyncio Tasks • Awaitable Objects • Cancellation and Shielding • Await on Multiple Objects • await with Timeout • Monitor Completion • await for the Earliest Result • Synchronization Objects • Queues • Subprocess • Interacting with Subprocess • Many Other Features
How to interface native resources in Python
Introduction • Interfacing Native resources • Using the Standard library • ctypes • numba • Embed C with CFFI • Building and Running CFFI • Compiling from C Header and Source Files • Building from a Shared Object File • Pointers and Structures • CFFI Build Script • ffi.new • cdef() – practical considerations • To distribute CFFI with PyInstaller • Cython • Cython Language • Primes in Python • Primes in Cython • Compile with Cython • Comparing Numba, Cython, and cffi • Compare the Speed • Summary
Deep dive into how to build Python wheels to integrate binary extension modules
Introduction • Package with Native Resources • Python Wheels • Pure vs Binary Wheels • Building binary wheels • Packaging Cython Extension • Packaging CFFI Extension • Cross-Distribution Linux Wheel • Manylinux • Manylinux Docker Images • Build a manylinux Wheel Manually • Supporting Multiple Platforms • Cross-Building Wheels • cibuildwheels
Common python tools and techniques to spot performance bottlenecks
Introduction • Profilers Technology • Top-Down Strategy • Timing Execution • Benchmark • cProfile • Sort by cumtime • Visualizing Report • Line profiling • Gathering Line Profiling Data • Line Profiling Result •Profiling Memory Usage • Line-by-Line Memory Profiling • Memory Profiling Tips & Tricks • Scalene • Scalene CLI report • Scalene CPU usage • Scalene Memory Usage • Optimization Loop • Micro-benchmarking • Choosing your Profilers • Bonus Annex
How to ship a Python Project
Overview • Distribution Package Lifecycle • PyInstaller • Dist Folder • PyInstaller Option • Source Distribution • Publishing Locally • Publishing on PyPI • testPyPI • Upload Preparation • Upload to testPyPI • Checking Project Page • Testing our Package • Uploading on PyPI • Automating Build • Going further
How to document your product and generate the documentation with Sphinx
Documenting API • Sphinx Overview • Sphinx Quickstart • Sphinx Configuration • Documentation Layout • reST markup • Generating Documentation • Writing Good Documentation
Some security considerations related to our Python product
Shopping on PyPI – The Challenges • Typosquatting Attacks • Looking for Candidates • Project Check List • Package Evaluation • Package Vulnerabilities Report • Securing Python Application • Python Security Cheat Sheet • Python Security Best Practice • Code Obfuscation
Complete an enquiry form and a Doulos representative will get back to you.
Enquiry FormPrice on request