Packman’s API

packman provides an API that can be used to easily create packages from external application (for instance, you could call packman from your build machine to generate packages after all tests passed).

The API is also usable when alternative implementations of get and pack for different components, as described here

Contents:

packman.packman.init_logger(base_level=20, verbose_level=10, logging_config=None)[source]

initializes a base logger

you can use this to init a logger in any of your files. this will use config.py’s LOGGER param and logging.dictConfig to configure the logger for you.

Parameters:
  • base_level (int|logging.LEVEL) – desired base logging level
  • verbose_level (int|logging.LEVEL) – desired verbose logging level
  • logging_dict (dict) – dictConfig based configuration. used to override the default configuration from config.py
Return type:

python logger

packman.packman.set_global_verbosity_level(is_verbose_output=False)[source]

sets the global verbosity level for console and the lgr logger.

Parameters:is_verbose_output (bool) – should be output be verbose
packman.packman.get_distro()[source]

returns the machine’s distro

packman.packman.check_distro(supported=('Ubuntu', 'debian', 'centos'), verbose=False)[source]

checks that the machine’s distro is supported

Parameters:
  • supported (tuple) – tuple of supported distros
  • verbose (bool) – verbosity level
packman.packman.get_component_config(component_name, components_dict=None, components_file=None)[source]

returns a component’s configuration

if components_dict is not supplied, a packages.py file in the cwd will be assumed unless components_file is explicitly given. after a components_dict is defined, a component_config will be returned for the specified component_name.

Parameters:
  • component (string) – component name to retrieve config for.
  • components_dict (dict) – dict containing components configuration
  • components_file (string) – components file to search in
Return type:

dict representing component configuration

packman.packman.packman_runner(action='pack', components_file=None, components=None, excluded=None, verbose=False)[source]

logic for running packman. mainly called from the cli (pkm.py)

if no components_file is supplied, we will assume a local packages.py as components_file.

if components are supplied, they will be iterated over. if excluded are supplied, they will be ignored.

if a pack.py or get.py files are present, and an action_component function exists in the files, those functions will be used. else, the base get and pack methods supplied with packman will be used. so for instance, if you have a component named x, and you want to write your own get function for it. Just write a get_x() function in get.py.

Parameters:
  • action (string) – action to perform (get, pack)
  • components_file (string) – path to file containing component config
  • components (string) – comma delimited list of components to perform action on.
  • excluded (string) – comma delimited list of components to exclude
  • verbose (bool) – determines output verbosity level
Return type:

None

packman.packman.get(component)[source]

retrieves resources for packaging

Note

component params are defined in packages.py

Note

param names in packages.py can be overriden by editing definitions.py which also has an explanation on each param.

Parameters:
  • package (dict) – dict representing package config as configured in packages.py
  • name (string) – package’s name will be appended to the filename and to the package depending on its type
  • version (string) – version to append to package
  • source_url (string) – source url to download
  • source_repo (string) – source repo to add for package retrieval
  • source_ppa (string) – source ppa to add for package retrieval
  • source_key (string) – source key to download
  • key_file (string) – key file path
  • reqs (list) – list of apt requirements
  • dst_path (string) – path where downloaded source are placed
  • package_path (string) – path where final package is placed
  • modules (list) – list of python modules to download
  • gems (list) – list of ruby gems to download
  • overwrite (bool) – indicated whether the sources directory be erased before creating a new package
Return type:

None

packman.packman.pack(component)[source]

creates a package according to the provided package configuration in packages.py uses fpm (https://github.com/jordansissel/fpm/wiki) to create packages.

Note

component params are defined in packages.py but can be passed directly to the pack function as a dict.

Note

param names in packages.py can be overriden by editing definitions.py which also has an explanation on each param.

Parameters:
  • component (string|dict) – string or dict representing component name or params (coorespondingly) as configured in packages.py
  • name (string) – package’s name will be appended to the filename and to the package depending on its type
  • version (string) – version to append to package
  • src_pkg_type (string) – package source type (as supported by fpm)
  • dst_pkg_types (list) – package destination types (as supported by fpm)
  • src_path (string) – path containing sources from which package will be created
  • tmp_pkg_path (string) – path where temp package is placed
  • package_path (string) – path where final package is placed
  • bootstrap_script (string) – path to place generated script
  • bootstrap_script_in_pkg (string) –
  • config_templates (dict) – configuration dict for the package’s config files
  • overwrite (bool) – indicates whether the destination directory be erased before creating a new package
  • mock (bool) – indicates whether a mock pack will be created (for testing purposes. does not use fpm)
Return type:

None

packman.packman.do(command, attempts=2, sleep_time=3, accepted_err_codes=None, capture=False, combine_stderr=False, sudo=False)[source]

executes a command locally with retries on failure.

if a command execution is successful, it will return a fabric object with the output (x.stdout, x.stderr, x.succeeded, etc..)

else, it will retry an attempts number of attempts and if all fails it will return the fabric output object. obviously, attempts must be larger than 0...

Parameters:
  • command (string) – shell command to be executed
  • attempts (int) – number of attempts to perform on failure
  • sleep_time (int) – sleeptime between attempts
  • capture (bool) – should the output be captured for parsing?
  • combine_stderr (bool) – combine stdout and stderr (NOT YET IMPL)
  • sudo (bool) – run as sudo
Return type:

responseObject (for fabric operation)

class packman.packman.CommonHandler[source]

common class to handle files and directories

find_in_dir(dir, pattern, sudo=True)[source]

finds file/s in a dir

Parameters:
  • dir (string) – directory to look in
  • patten (string) – what to look for
Return type:

stdout string if found, else None

is_dir(dir)[source]

checks if a directory exists

Parameters:dir (string) – directory to check
Return type:bool
is_file(file)[source]

checks if a file exists

Parameters:file (string) – file to check
Return type:bool
touch(file, sudo=True)[source]

creates a file

Parameters:file (string) – file to touch
mkdir(dir, sudo=True)[source]

creates (recursively) a directory

Parameters:dir (string) – directory to create
rmdir(dir, sudo=True)[source]

deletes a directory

Parameters:dir (string) – directory to remove
rm(file, sudo=True)[source]

deletes a file or a set of files

Parameters:file(s) (string) – file(s) to remove
cp(src, dst, recurse=True, sudo=True)[source]

copies (recuresively or not) files or directories

Parameters:
  • src (string) – source to copy
  • dst (string) – destination to copy to
  • recurse (bool) – should the copying process be recursive?
mv(src, dst, sudo=True)[source]

moves files or directories

Parameters:
  • src (string) – source to copy
  • dst (string) – destination to copy to
tar(chdir, output_file, input_path, opts='zvf', sudo=True)[source]

tars an input file or directory

Parameters:
  • chdir (string) – change to this dir before archiving
  • output_file (string) – tar output file path
  • input (string) – input path to create tar from
  • opts (string) – tar opts
untar(chdir, input_file, opts='zvf', strip=0, sudo=True)[source]

untars a file

Parameters:
  • chdir (string) – change to this dir before extracting
  • input_file (string) – file to untar
  • opts (string) – tar opts
class packman.packman.fpmHandler(name, input_type, output_type, source, sudo)[source]

Bases: packman.packman.CommonHandler

packaging handler

fpm(**kwargs)[source]
class packman.packman.PythonHandler[source]

Bases: packman.packman.CommonHandler

python operations handler

pips(modules, venv=False, attempts=5)[source]

pip installs a list of modules

Parameters:
  • modules (list) – python modules to pip install
  • venv (string) – (optional) if ommited, will use system python else, will use venv (for virtualenvs and such)
pip(module, venv=False, attempts=5, sudo=True, timeout='45')[source]

pip installs a module

Parameters:
  • module (string) – python module to pip install
  • venv (string) – (optional) if ommited, will use system python else, will use venv (for virtualenvs and such)
get_python_modules(modules, dir=False, venv=False)[source]

downloads python modules

Parameters:
  • modules (list) – python modules to download
  • dir (string) – dir to download modules to
  • venv (string) – (optional) if ommited, will use system python else, will use dir (for virtualenvs and such)
get_python_module(module, dir=False, venv=False)[source]

downloads a python module

Parameters:
  • module (string) – python module to download
  • dir (string) – dir to download module to
  • venv (string) – (optional) if ommited, will use system python else, will use dir (for virtualenvs and such)
check_module_installed(name, dir=False)[source]

checks to see that a module is installed

Parameters:
  • name (string) – module to check for
  • dir (string) – (optional) if ommited, will use system python else, will use dir (for virtualenvs and such)
venv(venv_dir, sudo=True)[source]

creates a virtualenv

Parameters:venv_dir (string) – venv path to create
class packman.packman.RubyHandler[source]

Bases: packman.packman.CommonHandler

get_ruby_gems(gems, dir=False)[source]

downloads a list of ruby gems

Parameters:
  • gems (list) – gems to download
  • dir (string) – directory to download gems to
get_ruby_gem(gem, rbenv=False, dir=False)[source]

downloads a ruby gem

Parameters:
  • gem (string) – gem to download
  • dir (string) – directory to download gem to
class packman.packman.YumHandler[source]

Bases: packman.packman.CommonHandler

static update()[source]

runs yum update

check_if_package_is_installed(package)[source]

checks if a package is installed

Parameters:package (string) – package name to check
Return type:bool representing whether package is installed or not
downloads(reqs, sources_path)[source]

downloads component requirements

Parameters:
  • reqs (list) – list of requirements to download
  • sources_path – path to download requirements to
download(package, dir, enable_repo=False)[source]

uses yum to download package debs from ubuntu’s repo

Parameters:
  • package (string) – package to download
  • dir (string) – dir to download to
installs(packages)[source]

yum installs a list of packages

Parameters:package (list) – packages to install
install(package)[source]

yum installs a package

Parameters:package (string) – package to install
add_src_repos(source_repos)[source]

adds a list of source repos to the apt repo

Parameters:source_repos (list) – repos to add to sources list
add_src_repo(source_repo)[source]

adds a source repo to the apt repo

Parameters:source_repo (string) – repo to add to sources list
add_keys(key_files)[source]

adds a list of keys to the local repo

Parameters:key_files (string) – key files paths
add_key(key_file)[source]

adds a key to the local repo

Parameters:key_file (string) – key file path
class packman.packman.AptHandler[source]

Bases: packman.packman.CommonHandler

dpkg_name(dir)[source]

renames deb files to conventional names

Parameters:dir (string) – dir to review
check_if_package_is_installed(package)[source]

checks if a package is installed

Parameters:package (string) – package name to check
Return type:bool representing whether package is installed or not
downloads(reqs, sources_path)[source]

downloads component requirements

Parameters:
  • reqs (list) – list of requirements to download
  • sources_path – path to download requirements to
download(package, dir)[source]

uses apt to download package debs from ubuntu’s repo

Parameters:
  • package (string) – package to download
  • dir (string) – dir to download to
autoremove(pkg)[source]

autoremoves package dependencies

Parameters:pkg (string) – package to remove
add_src_repos(source_repos)[source]

adds a list of source repos to the apt repo

Parameters:source_repos (list) – repos to add to sources list
add_src_repo(source_repo)[source]

adds a source repo to the apt repo

Parameters:source_repo (string) – repo to add to sources list
add_ppa_repos(source_ppas)[source]

adds a list of ppa repos to the apt repo

Parameters:source_ppas (list) – ppa urls to add
add_ppa_repo(source_ppa)[source]

adds a ppa repo to the apt repo

Parameters:source_ppa (string) – ppa url to add
add_keys(key_files)[source]

adds a list of keys to the local repo

Parameters:key_files (string) – key files paths
add_key(key_file)[source]

adds a key to the local repo

Parameters:key_file (string) – key file path
static update()[source]

runs apt-get update

installs(packages)[source]

apt-get installs a list of packages

Parameters:packages (list) – packages to install
install(package)[source]

apt-get installs a package

Parameters:package (string) – package to install
purges(packages)[source]

completely purges a list of packages from the local repo

Parameters:packages (list) – packages name to purge
purge(package)[source]

completely purges a package from the local repo

Parameters:package (string) – package name to purge
class packman.packman.WgetHandler[source]

Bases: packman.packman.CommonHandler

downloads(urls, dir=False, sudo=True)[source]

wgets a list of urls to a destination directory

Parameters:
  • urls (list) – a list of urls to download
  • dir (string) – download to dir...
download(url, dir=False, file=False, sudo=True)[source]

wgets a url to a destination directory or file

Parameters:
  • url (string) – url to wget?
  • dir (string) – download to dir....
  • file (string) – download to file...
class packman.packman.TemplateHandler[source]

Bases: packman.packman.CommonHandler

generate_configs(component, sudo=True)[source]

generates configuration files from templates

for every key in the configuration templates sub-dict, if a key corresponds with a templates/configs key (as defined in definitions.py) the relevant method for creating configuration files will be applied.

Parameters:component (dict) – contains the params to use in the template
generate_from_template(component_config, output_file, template_file, templates='package-templates/')[source]

generates configuration files from templates using jinja2 http://jinja.pocoo.org/docs/

Parameters:
  • component_config (dict) – contains the params to use in the template
  • output_file (string) – output file path
  • template_file (string) – template file name
  • templates (string) – template files directory