Func, MCollective, Salt and RunDeck

December 1st, 2011 | Posted by admin in Editorial

Func, Salt, MCollective and RunDeck

Communication and remote execution / command dispatching tools such as Func, Salt, MCollective and RunDeck have some overlap with the purpose of their usage. Depending on preference and the purpose for using such a tool, one may suit better than others.

Remote execution is an often underestimated component of cloud computing. Some compare it to the foundation of a building that easily gets looked over, but facilitates the construction of the edifice placed on top. The goal of a remote execution system is to have a means to execute commands on large numbers of desktops/servers without having to log into them one at a time.

Several open source cloud communication / remote execution solutions are currently available; we’ll just focus our attention on Func, MCollective, Salt and RunDeck. While differing in their strengths, there is enough similarities to review these four solutions together.

Func

Func is an acronym for “Fedora Unified Network Controller“, and is a popular open source tool that provides a two-way authenticated system for remote administration. Func is customizable with modules and easily works in conjunction with applications.

Integration with existing management tools
Func is included in Fedora and EPEL. Func is compatible with EL-4 and up, and Fedora 7 and up. Active development with Taboot, OpenSymbolic, Genome, FuncWeb, Loki, Funcshell, CAS and OpenNode. Puppet certificate authority integration.

Programming interface
Python API for module creation, development and customization. Various libraries exist to speak with Func from other languages (such as the Java/Groovy Binding). Func works over XMLRPC and SSL.

Scale it can handle
Can handle large networks of desktops/servers with a minimum of hassel and setup.

Ease of deployment
Simple setup: yum/rpm; debian, and AIX (limited).

Features offered
Manage an arbitrary group of machines all at once. Automatically distribute certificates to all “Slave” machines. Command line for sending remote commands and gathering data. Anyone can write their own modules using the Python module API. Works over XMLRPC and SSL. Uses certmaster for secure master-to-slave communication.

Resources
Func website
Func-list
#func IRC on irc.freenode.net

Salt

Salt is quite flexible and has shown appeal to simple system administrator tasks, as well as the DevOps community, up to application development. Salt is intended not only to be an interface as an administrative tool, but also a development platform. Salt has a very rich feature set, from being extremely fast to being highly extensible.

Integration with existing management tools
Salt is fundamentally detached in the way that it executes commands, which makes it easy to actively integrate into existing applications. Salt is integrated in the package managers for Red Hat, Fedora, Debian, Ubuntu, FreeBSD, Gentoo and Arch Linux. Salt is also integrated into libvirt for virtual machine management, as well as the network stack, and user management system for Linux and FreeBSD. Salt modules enable platform specific application integration.

Programming interface
Salt comes with a Python API. The api can be used to do many things, from integrating an application into the remote execution system, to gaining direct access to the system that integrates application interfaces.

Scale it can handle
Salt is engineered to handle a high load and has been tested with concurrent complex commands running simultaneously on almost 2000 clients at once for sustained periods. Salt also scales in the form of process space, as it supports multiple worker processes. Additionally, Salt can also scale from a topology perspective, the salt syndic interface allows for many groups of salt minions to be joined together in ad hoc management layouts.

Ease of deployment
To deploy Salt, it only needs to be installed, pointed to the salt master (a one line config change) and turned on. Generally, you can have Salt up and running in a matter of minutes.

Features offered
Salt integrates configuration management with remote execution, creating a complete control interface. A few of the built-in features include support for: Apache, Portage, FreeBSD, Tomcat, YUM. Salt also executes calls on SELinux and Puppet routines.

Resources
SaltStack website
IRC channel #salt on OFTC
Google User Group
SaltStack video channel

MCollective

MCollective, formerly known as “Marionette Collective” (acquired by Puppet Labs in 2010), is a tool primarily used for programmatic execution of System Administration actions on clusters of servers. MCollective delivers a very scalable and very fast parallel execution environment by leveraging Publish Subscribe Middleware and meta data (as opposed to hostnames).

Integration with existing management tools
MCollective’s core framework allows the user to build out a customized system (replace MCollective’s choice of middleware – STOMP compliant middleware – with your own like AMQP based). You can replace the authorization system with one that suits your local needs. Additionally you can replace the serialization – Ruby Marshal and YAML based – with your own like JSONSchema that is cross language. You can also add sources of data, like Chef and Puppet, and can provide a plugin to give access to your tools data. You can also create a central inventory of services leveraging MCollective as transport that can run and distribute inventory data on a regular basis.

Programming interface
Write simple RPC style agents, clients and Web UIs in Ruby.

Scale it can handle
Small to very large clusters of servers. Re-use the ability of middleware to do clustering, routing and network isolation to realize secure and scalable setups.

Ease of deployment
Setup is straight-forward, requiring: 1) A Stomp server, tested against ActiveMQ; 2) Ruby; Rubygems; 3)A Ruby Stomp Client. Offers serveral easy installs: yum/rpm, plus a blip.tv channel with videos, webinars and screen casts for additional support.

Features offered
Interact with small to large server clusters. Uses a broadcast paradigm for request distribution. Address machines by meta-data (from Puppet, Chef, Facter, Ohai or your own plugin) as opposed to their hostnames. Simple to use command line tools to call remote agents. Write custom infrastructure reports. Active community developing agents to manage packages, services and other components. Write simple RPC style agents, clients and Web UIs in Ruby. Extremely pluggable and adaptable to local needs.

Resources
MCollective website
Author’s blog
MCollective User Group
Video Channel
@puppetlabs

RunDeck

RunDeck’s primary goal is to play the role of a job console, a web-based simple workflow tool for DevOps teams to define and run a variety of routine jobs in a distributed environment.

Integration with existing management tools
There exist plugins for: Chef, Puppet, Amazon EC2, and Jenkins. RunDeck has a few plugin interfaces for retrieving Node resources and Job Option values from other sources. Plugins can be defined using shell scripts which allows hand-rolled integration with e.g. in-house CMDBs.  It also has webhooks for reporting job results to other systems, and a Web API for interacting with jobs and adhoc commands, resources, and executions. 

Programming interface
There is a HTTP API to perform most tasks that are available via the GUI.  Most plugins can be written as shell scripts and all can be written in Java. 

Scale it can handle
Some clients have used it with upwards of 7000 nodes.  Depending on what kind of scale (concurrent jobs vs. number of nodes), different RAM requirements come into play. 

Ease of deployment
There are several easy installs: yum/rpm; debian; and a single-file “launcher” jar which can install and run itself (similar to Jenkins’ war). Projects can be set up and configured via the CLI or GUI. Requirements: java 6 (works with openjdk).

Features offered
There is a job scheduler for running commands and workflows on multiple nodes. The GUI provides easy shared view of live executions, history, nodes and jobs. Also provides CLI and web API. Authentication and Authorization via LDAP/Active-Directory or file-based. Comprehensive access control policy: control who can see, run, and modify Projects and Jobs. Adhoc executions in the GUI; type, run, and watch. Retry a job/adhoc command on any failed nodes. Plugin interface for Node sources and definition formats. Job and Node definitions can be imported/exported in XML or YAML for use with SCM systems.

Resources
RunDeck website
Google User Group
Videos/Screencasts by Alex Honor and Greg Schueler
@Rundeck

You can follow any responses to this entry through the RSS 2.0 You can skip to the end and leave a response. Pinging is currently not allowed.

6 Responses



Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>