Biography

Principal Architect, building DS pipelines and architectures to process and analyze big amounts of data.

Workshop

From Script to Pipeline: Long Road and Shortcuts

There is a long path between a Python script and a production solution. How to start some new open source project? How can you build it into a proper package and make this complex CI/CD pipeline to pick it up and deploy automatically on some staging environment?

I propose us to go from a basic Python script to a package, then from package to a CI/CD project on Jenkins (let’s deploy it automatically as well). And all this goes further up to a deployed version somewhere that you can look at not so long after you’ve merged your last Pull Request on GitHub.

This workshop is a tutorial giving some illustration on what IaaC really is, and how you can automate all different steps on this path to your own pipeline.

Part 1: Intro to Python packages

• Typical structure of a good Python project
• What do you need to convert your Python code into package
• How to manage Python packages inside your infrastructure

Part 2: Jenkins deployment with Ansible

• Configuration management and Roles
• Configuring Jenkins Role to our needs
• Security and credentials management for GitHub integration
• Deploying ready-to-work Jenkins with just one command

Part 3: Jenkins jobs and IaC

• Using Groovy + Gradle to write job DSL
• Production pipelines for Python projects
• How to further organize your workflow

The main goal of this workshop is to help participants to go through the steps that convert a basic Python script into a production package and setting a full-featured CI/CD pipeline to maintain its development cycle.

The target audience includes people who are interested in IaC concept as well as building CI/CD pipelines from scratch. Python is used as an example of a popular programming language of choice, but the rest of the workshop is applicable to other kinds of projects as well.

– Installations:

                – Python 3.5+ locally on a machine that will be used for running deployment scripts

                – Cloud-based (or local) VM preferably with Ubuntu 16.04+ (CentOS 7 may also work) with Python 3.5+ and Java 8 installed (should work with both OpenJDK/Oracle)

– Technical knowledge:

                – Basic programming in Python or some other programming language

                – Basic SSH/shell usage