Jenkins or: How I learned to stop worrying and love automation

hopefully you're here for this

presentation on Jenkins and not for

something else a little bit about myself

I have been working with Drupal since

also write quite a bit I write blog

posts I wrote a book I actually have a

few little coupon things these say on

them 25% off but I made the coupon 100

percent off so throughout the

presentation I'll try to try to give a

few of these Cathy's out the cool thing

about the way that I'm publishing this

book it's only in pub so anytime that

there's updates danceable I update the

book the book has complete automated

test coverage as well so I update the

book examples and then you get them for

free it's a nice nice model I also call

myself an Automator of things because

that's kind of what I do

aqueous is what I do for my personal

projects I wish I could do that for

things like gardening because everything

that I grow at my house ends up dying a

year later or a week later but for the

first one of these who knows the

reference in this picture and in the

title okay well someone else what's the

what's the title of the movie

yeah dr. strange like and that's

what's-his-name all right so with this

this presentation is going to be kind of

simple but towards the end I figure I

want to give everybody a good kind of

broad overview of Jenkins because some

people here who here has ever used

Jenkins at all so there's there's a good

number a lot of people never used it

some people might have like used it on a

project that they never set it up and

they never configured it so I want to go

from like the beginning of what is it

how do you use it how do you set it up

and then get into how to use it with

Drupal and then finally at the end we

can kind of get deeper into things if

you want to see some of the examples and

stuff like that so a little history on

the project originally a long long time

ago it was called Hudson and it was

developed by a guy at Sun Microsystems

unfortunately at some point in the

history a big company called Oracle

bought Sun Microsystems and they started

screwing with all these open-source

projects that were amazing and that led

to some internal strife in the Hudson

team and the guy who created it

eventually Oracle says he forked it he

says he renamed the project and Oracle

Ford did but basically they came up with

a new name and it was called Jenkins

so eventually many years later Jenkins

came under the purview of a company

called cloud B's it's still completely

open-source it's still primarily a Java

based tool and it's like if you use Java

it's like the shoo-in for anything CI

but it's it's a Java based tool that

that is well maintained has constant

updates has a very large community

and it's under the stewardship of

cloudBees now for another one of these

copies of the book who can tell me

another popular software product that

Oracle has bought out as part of their

bio to son and kind of noisy boys I

think yeah OpenOffice there's there's a

number Sun Sun was highly productive and

Oracle is highly seductive I guess in

terms of money is great for causing of

working if you want to get your coffins

first project for then go to Oracle back

in the early 2000s I forget like 2004 or

something and when Hudson was originally

made it became super popular and son it

was open source it became super popular

everywhere because see I was a pretty

new thing at least open source see I was

a pretty little thing back then and by

like two thousand seven eight nine it

was basically the only open source CI

tool since Jenkins UI and paradigms and

all that kind of just stayed the exact

same from 2001 to 2011 twelve thirteen

all the sudden people realized oh people

want usability and people want to be

able to do things in language besides

Java so all these other things have

since come into being and some of them

integrate a little better a little worse

with different products like Travis CI

fees github travis circle CI some other

tools or like click a button and you're

all integrated jenkins a little bit more

work but it is open source and free for

private projects so that's nice

but there's a lot more tools today so

Jenkins has to be more competitive um

getting started with getting Jenkins

onto a server similar so you can start

using it a few of the considerations

that you need to make one is random

Jenkins is a Hungry Butler it needs lots

of RAM it's basically as much as you can

give it it will run on a one gigabyte

VPS or on like a one gigabyte virtual

machine something like that but it

doesn't run well and it will kill itself

when it hits out of memory issues even

if you're just doing things like cron

jobs and stuff Jenkins Jenkins likes

lots of RAM so two gigs four gigs

minimum requirement if you're doing a

lot of things a lot of Drupal sites you

might want to go eight gig 16 gigs or

also spread out jobs on multiple slaves

I'm not going to get into that in this

presentation CPU not usually that

important to have like super beefy CPU

on your server that runs Jenkins unless

you have jobs that do tons and tons of

work and PHP nodejs and stuff if it's

just building a project and running

tests it doesn't need to have something

crazy another thing that's important is

don't ever fill up the system disk

Jenkins doesn't handle that too

gracefully what it does is it just stops

talking jobs and it doesn't tell you

anything about it there might be a log

message but make sure that you monitor

these things so it has either just log

in to the server

on top or have a tool monitoring these

things and make sure that you're not

hitting your limits and if you are you

might need to upgrade your server also I

do run Jenkins a lot of times on a

really small server like a 512 maybe PS

just because it's super duper cheap and

if we didn't have it running we wouldn't

want anything so have something it's

better than nothing so in the

presentation which I'll link later

there's a shell script that I have that

I've run as a cron job on those smaller

servers which is on it just on github

gist that you can copy and paste and it

just make sure the service is running

and this projector does this in this

room I don't

so installing Jenkins it's really simple

right you just install Java installed

Jenkins and you're done it's just like

Apache Solr except Jenkins Jenkins is is

a lot more complex beast and one thing

that some people might not realize is

when you automate things with Jenkins

you put all your keys and API keys and

your SSH keys and all these things into

Jenkins if somebody gets controller it's

going to completely own and demolish

your infrastructure because they have

the keys to the kingdom so of the people

who have ramas Jenkins server have you

ever seen a Jenkins server look like

this after a year or two anybody I have

I know pretty much every Jenkins server

I've ever seen ends up like this the

problem is people don't update it people

don't maintain the server people don't

have any sort of testing for things they

do so they add in a plugin and it messes

everything up and when their Jenkins is

hosts forever I've been at a lot of

organizations where so they have like a

list of their Jenkins servers and then

there's a column that says has it been

hacked yet and like 90% of them are yes

because they're running like a three

year old version of Jenkins because

everybody's scared of upgrading because

they have these plugins that are three

years old that all their stuff is

dependent on but the server's hacked but

it still has to run because they're

dependent on it to do their business

operations it's not a good situation

it's I mean it's like a Drupal site if

you I know people who have not updated

their Drupal sites from seven point six

or eight or whatever it is Phenix Drupal

I mean if you're running Drupal 6 and

you have drop garden my drop guard job

my drop lizard that's it you that I

would still get into Drupal 7 as soon as

you can but but it's it's the name of

the game and software is updating so

there's the way that I recommend

installing Jenkins and the way that I do

it is you want it to be secure first of

all because if you're like let's say

you're on the wonderful open Wi-Fi here

and you log into your Jenkins server and

it's on HTTP anybody with a laptop here

could immediately pick up your login

credentials get into your Jenkins server

and do anything that you could including

like deleting a code base or deploying

new code or changing the checked out of

branch on a Drupal site or if it's a

more boring server just running your via

tests and seeing how cool that yeah yeah

VPN is an option but I for public server

like a lot of times it's a group of

people that's remote or something's I'm

working with so I have nginx and let's

encrypt let's encrypt generates a cert

and keeps the cert active and nginx

proxies little traffic the Jenkins and

you can also do some other things to

make the installation a little bit more

secure with that but in general having

it behind the HTTPS only it's nice and I

think that because it's still recording

believe me but it just keeps hiding

itself on the screen engine X is just a

proxy so the other thing that I

encourage is Jenkins as a CI tool and

you're going to start using CI as a

cultural thing it's your company or for

your projects wouldn't be good to have

CI for your CI so that you can actually

make sure that your testing and your

integration and your deployments work

before you make updates to all that

stuff so I always make sure that there's

actual CI for the Jenkins instance so I

don't have

and I'm just deploying update to Jenkins

and hope you know as I think Chris Urban

yesterday and our other presentation

what was the code like cross your

fingers pray

if it if it works it's in prod and it's

done that's not how that's not how I

want my Jenkins server drone especially

if I'm dependent on it for code deploys

or 4 4 code builds things like that

don't look dumb luck yeah I don't want

dumb luck

managing my Drupal sites so and you know

you wouldn't develop a Drupal site in

production would you so why would you

develop all of your automation in

production that sounds like a recipe for

disaster

if you run a job and you accidentally

had the wrong parameter and you had prod

in where it should be dev all sudden

your prod at an Armin gets locked off

and for all them bad things can happen

so you want to be able to test things

locally and to run Jenkins locally and

make changes locally and then put them

on to production in an automated fashion

repeatedly if you want to skip ahead

about a thousand steps I have a github

repo that I've built for this

presentation unfortunately I was going

to do a few more things with it but I

ended up in the hospital a couple weeks

ago so I did not get to do a couple of

those things but it's pretty complete

and I'm gonna keep working on it after

this because I'm actually now using it

to manage some of my Drupal sites

because better than the old Jenkins

server setup that I was easy

if you want to do something that's a

little easier for Jenkins management you

can there is a platform or product

business service service as a service I

don't look CIA as a service through

fabi's so you can get all the same stuff

with Jenkins but managed hosted by them

with some of their security and tools

and things and the guarantee that it's

always up to date they have you have

engineers available it's a bit more

expensive though I'm having an open

source tool it's kind of the same thing

it's like if you ran if you're on their

site through Squarespace a Twix you're

paying a lot more than if you use Drupal

which is free but you're responsible for

Drupal updates and stuff like that but

they have it's basically the name of the

game if you want hosts of Jenkins if you

want to use it right now during this

presentation and stop listening to me

you can use docker this command and

that'll bring up their their basic like

the latest demo

docker container running Jenkins with

blue ocean talk a little bit more about

the ocean in a bit so you have Jenkins

setup let's say using that docker thing

or using the linked linked project that

I was working on once you're in there's

a few things that you might need to

configure some of them are more pressing

than others but this is another reason

why I like doing it locally before I go

to production because if you build a

Jenkins server in production at least in

the old days it used to not even be

protected there was no admin login by

default so you to install Jenkins and

then if somebody else got to it before

they could lock you out of it that's

never a good thing

or sometimes you install Jenkins you log

in and you change a 70 minute

you have that's not good either so it's

good to do this locally first get your

get your feet wet the first thing that

you'll probably want to do and it's

something that you'll do forever

basically when you're using Jenkins is

start evaluating what plugins are

necessary for your work plugins are very

much like Drupal modules and that's a

good thing and a bad thing just like

this Drupal modules that's a good thing

and a bad thing some plugins break

everything every few months other

plugins are stable and reliable and

awesome all the time but you'll end up

using at least 5 or 10 plugins sometimes

instance to help make things faster

easier more convenient and plugins do

everything from like making it so you

can interact with github directly so you

can have commits on github trigger

builds and Jenkins or do things like put

timestamps in your build log so you can

see how long things take that kind of

stuff one of the biggest plugins Suites

which is slowly becoming basically I'm

guessing with like Jenkins 3.0 will be

this called blue ocean and like I said

earlier Jenkins was the name of the game

and the state of the art in 2004-2005

like two thousand nine ten eleven but a

lot of people started realizing that

some of these other new tools circles

yeah get live CI and Travis CI and all

these other ones some of some of them

even open-source they were a lot more

usable they were a lot more user

friendly and also did things like

instead of having to click through an

interface to build a job you could just

throw a file in your repository and then

when you added it to the system it would

start building it automatically Jenkins

has that now through blue ocean blue

ocean and pipelines but there's still

the the major effort went into it and

what was it 20 14 15 16

and it's still not the default so the

default is still the old ancient

interface which they have retina

graphics which is nice but it's still

this ancient like if you used Hudson and

Jenkins today a lot of it is just

exactly the same just nicer icons so

blue ocean is just a complete rewrite of

the interface and it's a lot more usable

it's a lot more friendly towards the

people that will actually build the jobs

like the old Jenkins interface is really

friendly for the people that build your

automation but you should if you build

automation rightly you're doing it like

for two percent of the time and you're

using it like 98 percent of times so

it's better to target the 98 percent so

there's there's an issue if you want to

follow it it's linked in here eventually

it's going to be the default so instead

of logging into Jenkins and getting that

old interface you get the new one and

that'll be nice because the only way to

do that now is to use nginx and to

rewrite requests to slash blue and

that's kind of annoying to me another

thing that's important in configuring a

Jenkins server because generally

speaking and I would recommend this

Jenkins should be completely separate

from like your Drupal server or any

other service that you're running for

many reasons isolation and the fact that

I'll get to it in a minute but basically

you're going to be putting things into

Jenkins to let you be able to access

other tools and code bases and things so

one thing to keep in mind there is if

you can don't give it the keys of the

kingdom give it the keys that it needs

just to do it it needs to do so like if

it just needs to check out the code base

on github and deploy it you don't need

to give it push access to your github

codebase so use a github deploy key with

read-only access

don't use your like personal key that

can do anything to any github repo that

you own seen that happen a lot of times

and that's an easy way to get hacked API

keys and things like that usually what

you would do is store them in the

Jenkins home directory

there's other ways to do it you can also

use key storage systems and things but

there's some plugins for it but it gets

kind of complicated and doesn't work

with everything so one thing to keep in

mind is that you need to be careful and

vigilant about who has admin level

access to Jenkins because all these

things basically their files on a file

system that somebody could read if they

get access let's see and this is the

interface that comes with Jenkins for if

you need to add like an SSH key you can

either enter it in directly and it

stores it in a little file in the

Jenkins home folder you can have it

stored somewhere on the Jenkins server

and it could read it in you can also add

usernames and passwords and API keys and

things and then Jenkins gives these

these credentials to any job that needs

them there's ways to make them more

limited so you can like the credential

that only these jobs can use in a

credential in these jobs use but that's

a little more complicated than I'm going

to get into today and like I said

earlier be careful who has access to

Jenkins as basically if you have if you

have a default Jenkins instance you just

installed it and you create a new user

account that new user account has all

privileges and rights just like you do

so basically unlike Drupal where every

new user just has like admin area

authenticated user and it has like two

permissions and Jenkins every user gets

basically two Jenkins and so if you have

files in your Jenkins home that have

credentials in them that new user could

write a Jenkins job that cats a file out

and then they could grab your keys and

do things with it so another thing

that's important in a Jenkins server is

making sure that you have a good access

control

so the way that I usually do this is

using the role strategy plugin and if

you use Drupal this is like I don't know

it's like halfway between what Drupal's

groups and the users are not groups what

is it called

is it groups roles you have permissions

and roles

it's kind of like drupal's permissions

and roles but it can be a little bit

more daunting at first because it's like

you want to do it by Jabra globally and

it's like that's the only option so you

can do it by job and like every single

job you have to configure who has access

to read/write see the workspace run it

see the builds all that kind of stuff

another thing to be careful of is the

workspace itself when you're when you're

setting up permissions for a Jenkins

server if somebody has access to the

workspace and part of your code build

job does something that like writes a

file to the code base that has an API

key or access key in it and then it puts

that code on to your production server

or something like that

anybody with read access to the

workspace could technically get at that

key so one thing I usually do is I have

there's a plug-in called the workspace

cleanup plug-in it deletes the workspace

after a job sometimes that's not

something you want to do actually but

you have to be careful about who has

access to the workspace for anything

that would do something that that writes

secrets to the workspace got a quick

drink

backing up Jenkins is actually

surprisingly easy Jenkins does

everything under the Sun inside its

Jenkins home folder there are very

specialized advanced to use cases where

that's not the case but 99.9 percent of

the time everything that you need in a

Jenkins installation is inside Jenkins

home so you literally just zip that home

folder put it somewhere and you have a

backup so what I usually do is I just I

have an Amazon s3 bucket it cost like

seven cents a month to have you know

like 60 days of backups unless we have

gargantuan code bases that you're

backing up which is a really bad idea

and get a reason that clean up your

workspaces but anyway there's I have an

ant scible playbook I think this is let

me click on it it's an en civil playbook

that grabs the Jenkins home it makes

sure there's a bucket name bucket named

Jenkins backup in my case it sinks the

contents there's actually an ansible

module for this but Amazonas s3 sync was

causing your tissues and then it makes

sure that it's there and for sadly

here's where your backup is so that's

nice and that's that's what I usually do

is I put that playbook on the server and

the Jenkins home folder and then I have

a Jenkins job that runs every night and

it just does that and it zips it up with

a timestamp and if your Jenkins home is

small it's like I said like pennies to

do backups on its three if your Jenkins

home is huge you might want to change

your change the way that you use the

backup for you - I think that script

actually excludes workspaces but you'll

have to see in your own a decision

so finally jenkins and drupal that's why

we're here what are some things that we

can do with jenkins in drupal well

anything that you do manually you can do

it this jenkins anything that you do in

Travis CI or other things you can do it

with Jenkins to some of the most common

things are on here running tests

building dependencies so a lot of people

use Jenkins like Acquia we have a tool

called BLT build and launch tools and it

actually by default uses Travis CI but

you can use it with Jenkins and it in

your codebase you just have a composer

JSON your tests and custom code like

that's all you have in your codebase so

your codebase could be like 100

kilobytes you don't have to include

Drupal core all the modules and patches

and all that junk and then what blt does

is every time you commit code it builds

the code base on Travis or Jenkins or

wherever and then it runs your tests and

then it does everything and if you have

it configured it'll also deploy it to

your development environment so that's

one thing that a lot of people use

Jenkins for because it's really annoying

to have to do that my hand and your

options are basically either somebody

does that by hand every time you deploy

code or you automated in Jenkins and you

don't have to worry about it ever again

another thing that Jenkins can be useful

for is running cron a lot of people just

set up a cron job on a server or if

you're using a hosting provider you can

run cron usually in their interface

somewhere but you might need more

specialized control over the crowd or

scheduling Jenkins has pretty flexible

scheduling you can also set it so that

like cron could be run after certain

things happen so a lot of people who

need more advanced things done with

crime end up running it through a tool

like Jenkins instead you can also deploy

new sites so any example that I'm giving

I'm doing a multi-site install so I had

I I didn't this is one of the things I

didn't get to do do before today but you

can

have a playbook or script or something

that sets up the new database installs

Drupal and make sure everything's in

line for a new multi-site site or even

if you're using like Amazon route 53 you

could do everything like say I need a

new website I need Jeff Garlin's Jenkins

stuff com

just type in the website name and it

would do everything from giving you a

domain name to building this folder and

everything in your codebase pulling the

codebase building the database building

your site it's it's like a it could be a

performance way of getting like an

automated site builder tool doing

another thing that a lot of people do is

deploying updates so instead of just

automatically deploying things to dev or

stage or prod if you have multiple

environments like what I usually do is I

have one production server and in my

local environment for a Drupal site so

when I want to deploy something I commit

things to my code base and then I click

a button jenkins and it deploys it to my

production site instead of doing it

automatically I like doing it on my own

and so for a quick little demo I think

I'm doing a little bit fast but for a

quick little demo I have my my Jenkins

project set up and I'm gonna use it on

my Midwestern Mac that will see a

website which is on a server that's

actually running Drupal 7 I have one

that's running Drupal 8 but there's only

one side on it so it didn't get to test

like multi-site stuff and it has six

different sites on and here's a quick

demo of how I have that set up using

this this installation so let me go

quickly to give

drupal jenkins multi-site so this is the

project i was mentioning earlier and it

has as i said it you can install locally

so you can test things out and build

things locally before you go to

production it also has a production set

up so that you can use like VPS anywhere

as long as you have root access you can

do it and it will build everything the

same way in production as you have on

local the local set up uses docker and a

fun thing i just found out last night is

that you can actually run docker

containers inside docker containers so

like i can do all my CI stuff inside

docker

that uses doctor and i didn't think I'd

be able to do that so that is cool and

it has an MIT license so if you want to

take this code and completely modify it

for your needs go ahead and fork it I'm

completely happy with that and this this

setup depends on an antipope playbook

that uses a bunch of money ansible rules

so there's a lot of these girl and guy

roles but it sets up a bunch of

different things for my Jenkins server

that I use as tools to help me automate

a little easier so I'm using ansible

I'll show you how to use that in a

minute a cert bot is the tool that gets

certificates from let's encrypt I use

docker containers to do things like

build compose dependencies that way I

don't have to install PHP and composer

and all these other tools on the Jenkins

server I try to limit the Jenkins server

to just like Java Jenkins nginx thing as

small as possible now there's some other

tools you might be like git but keeping

it keeping the attack surface small on

the server is a very good thing when you

have the keys of the kingdom on your

server

so then there's there's Java and Jenkins

of course Java is required to run

Jenkins and then I have get nginx and

Gen X which is just used to serve the

SSL for the site basically it's a

reverse proxy then I have a security

role that that does what I called it and

everybody calls it the first five type

setup steps it locks down SSH access it

installs steal failed to ban so that if

somebody tries for brute force in your

server though this block some of those

kind of things

sets up automated security updates for a

month two or more sent to us and then I

got a firewall that's it's just sets up

IP tables rules for so that you can only

get access to like Jenkins and that's it

and SSH and then pip is used with

ansible to do a few of the automation

steps so once you get all that setup it

installs the Jenkins server and I had

enough time to get these four jobs done

just to show you a sample of some of the

jobs that I would build personally my

convention is always they have snake

snake case is it

yeah not camel case it's snake case

where it's all lowercase with under

lines instead of spaces because when

you're automating things with Jenkins

you can use spaces but spaces cause all

kinds of weird issues so I use this kind

of format for jobs other people don't

but as a simple example

I actually made an update to my code

base that multi-site code base if I go

to midwestern mat.com I go to available

updates

I find that honey pots out-of-date so I

just grabbed the module this is a Drupal

downloaded the module and drugged and

drugged it into my codebase

it was very barbarian feeling but anyway

so I did all that and I committed it now

I'm going to deploy that code since this

is a multi-site there's six Drupal sites

but there's one code base so code deploy

is just literally putting that code on

the server and then running updates and

clearing caches for each of those sites

so I'll show you how that's I'll

actually let's do it first

so Jenkins gives you a nice live view of

what's going on on the server and gives

you counsel out but and I use a plug-in

to give timestamp so I can see how long

things take and you can see it deployed

the code before this was the commit on

the server and after that was to commit

so that's all nice and good and now it's

going through all the sites on the

server midwestern Mac Jeff Garlin calm

carry of ours before I do SDL so it's

going through all these sites and it's

running database updates and then it's

clearing caches each one using brush and

I'll show you how that's done in just a

minute once it's finished

there we go and at the end it's always

wonderful to see that success so if I go

back to the project I can show you how

does this set up I used to do it like

for something simple like this I used to

do a lot of things using like an SSH

wrapper for a command or something but

that was always more complicated things

always got messed up so now I use

ansible I call this the formatting

sensible where you're not actually using

like ansible playbooks you just

literally use ansible to run a command

on the server and the way this is set up

is there's what ansible calls an

inventory file it says MN for midwestern

Mac and then is here it just tells you

here's the server it's out there

midwestern Mac comma or whatever so I

say ansible and then I tell it what

servers to work on in this case just mmm

is the one server and then use the git

module and then these are arguments to

answer all to tell it what to do so it's

saying go to my git repository and then

check out the master branch the latest

version of master make sure it updates

to the latest commit and stick it into

this folder make sure that it's there so

a cool thing about the way that ansible

does this is it it actually will check

and see is that already done if it is it

will say changed equals false and it

will just tell you like it do anything

because it's already there but in our

case it did see enough they say they saw

the commit has changed and it said

changed equals true and then I have just

an array here of all the different

websites that are on that multi-site and

it loops through the array and runs

brush-up DB and drush CC all that could

have been a little more fancy and used

my gift aliases which are actually in

that code base but I didn't have this

Jeb used the code base kind of

intentionally cuz

few minutes so that's this is like super

simple and using ansible it makes it

really easy just run commands on a

remote server as long as you have SSH

access to the server and for if it's

local like if you did do something

against my recommendation and run

Jenkins and Drupal on the same server

which would often end up in the Jenkins

blowing up or Drupal blowing up then you

wouldn't even need to use ansible and

couldn't see stress directly but I don't

anyway so that's how that works and then

there's another couple jobs here cron

run this is just to show how simple it

is to run from a remote server with

Jenkins I use the same thing ansible

operate on the midwestern maxover and

then if you don't provide so ansible sit

like this is a module if you don't

provide a module it defaults to the

command module which literally runs the

string as a command on the server so I

just haven't run drush core cron and I

have a variable here website so I can

choose which of the six web sites it

runs crown on so if I go back here and

build it I can choose which website I'm

gonna run crown on Jeff Garlin back comm

go here and I can watch the output

and of course it goes with there we go

okay so if I go to Jeff Keeling that

comment should show that cron run ran

very recently I got a status report last

around 17 seconds ago so benefice oh and

I'll show you I'll prove that the

midwestern Mac update has been applied

if I get available updates honeypot

should be up-to-date that's good too so

it for something like cron like I said

earlier it's something that Jenkins can

automate and and have it have it run

like every five minutes or every hour or

whatever and there's a plug-in called

build periodically with parameters it's

the parameterised trigger plugin because

out of the box that comes with this

build periodically and you can just say

like run it every five minutes but if

you have parameters like which website

you might want to have like every five

minutes for Jeff gala calm but every day

for something else that doesn't get much

traffic so you can actually add

schedules for each parameter and it

gives you the syntax for that here

see so let me go to deploy to try and

run database backup is another one

that's I swear you can like if you're

using what's the module backup and

migrate freezing that which I also don't

recommend but if you are using that you

can have it like build a zip file if

your tar ball or something of your

codebase and database and then ship it

somewhere or you can used rush rush now

has the ability to build like your

entire code base and files in your

database and put it somewhere but I

usually backup everything separate so

this one just does the database it's

again very simple it has the selection

of the website as a parameter and then

it uses ansible first it does a rush

SQL dump and it compresses it and it

dumps it into a particular file and then

it copies it to this server so it copies

it from the from the destination like

from the web server to the Jenkins

server and then it deletes it from the

web server and then I didn't implement

this because I didn't get time to stick

my Amazon credentials for Jenkins into

here yet but the last step would be

copying it up to s3 and deleting it from

the server so that's another thing and

just like with the cron usually you'd

want to do that like daily or weekly or

whatever the schedule is that you need

let's see the last one so the last test

takes five minutes to run so I'm gonna

be kicking that one off and then getting

back to the end of my presentation I'll

show you all the wonderful magic

now there was a good presentation on

behalf that I highly recommend if you

need to know more about behavioral

testing by Steve Kirsh yeah it was a

little bit earlier today I'm sure that

Kevin Poole already has it up on YouTube

pretty much but this one this will take

fraud that Drupal um this production

demo website actually has like it's kind

of like my canary in the coal mine for

Drupal VM it's it's a live production

website that's running on Drupal VMs

stuff on a digitalocean droplet and the

code base has B hat tests and it has

integration with varnish and a bunch of

other things I'm kind of just using it

to make sure that everything I do and

hopefully em doesn't blow up everybody's

stuff all the time

which it hasn't because I have this

thing that tells me when it does so

that's nice but it comes with some B hat

tests and I have this job that will

schedule it and start it will do is it

actually uses Jenkins git integration

and it plumbs the repository into the

Jenkins workspace it it builds a docker

container which it's doing right now or

gold roof will be M it builds a docker

container it builds the codebase inside

the docker container using composer and

then it uses B hat to run you know then

it installs Troopa with josh then it

uses we have to run all the behavioral

tests so that every time that I commit

code I can have that we had to solve run

automatically

I didn't integrate this directly with

github so that every commit triggers a

build but you can do that if you're on a

public server if you're on your

localhost github can't call local host

because

or its local to you and not github so

anyway while this goes it takes a while

because downloading every dependency

ever well that goes I'll show you it

really quick how it works and then we'll

get back to the the tail end of the

presentation and open it up for us in

discussion so this one again is pretty

darn simple instead of using ansible

this time it's using docker to do

everything the first step is it

integrates with git so this is a github

repository you can actually clone it and

it includes as a bonus all of the

secrets encrypted with ansible vaults so

if you can crack ansible vault you could

have all of the keys to the kingdom for

Tripp will be on live they're all in the

codebase but I don't think you'll be

able to do that until you have a quantum

computer so so this uses if they get

integration to check out the master

branch you can also have it check out

multiple branches or check out a

different branch or have a parameter as

the branch name that kind of thing and

then it uses docker composed to build

that the built in docker compose file

that comes with Drupal VM and then it

installs the composer dependencies that

just uses composer install then it

installs the site using dress and then

it uses me hat to run the B hat docker

configuration which I can't scroll very

well it runs that we had docker

configuration which just tells it which

tests and where to look for things and

all that and once it finishes you'll be

able to see see the wonderful result of

that

so before you get to see that since it's

going to take a few minutes and I'm glad

I plugged in my computer because the CPU

is like 400% now I will go to the end so

one thing that I often run into is I'm

like man I just did that thing two times

I should spend like six hours automating

it sometimes actually do that and I

realized oh I've only never done that

three times and it takes like five

minutes

I shouldn't have spent six hours

automating it so of course there's this

wonderful xkcd comic that gives you a

good chart that's like when you're

thinking about automating something you

do with Drupal make sure that it's the

what is it the risk/reward not from

screw or make sure that that ends

justify the means

all right make sure that ROI is there

yes that's business speak I don't ever

remember anyways so you know a lot of

the things like running Drupal cron it's

a very quick thing if you want to go in

the interface into it but you're gonna

do it every five minutes through the

night you don't want to be up at night

clicking run crown on your Drupal site

so automated but if you you know it's

like restoring a database if you have a

big website you probably want to

automate that for safety's sake but if

you have a website that has very low

traffic I like some of the sites that I

want on my thing get like 12 visits a

day I don't need to have automation for

a database or store I can just be like

oh the websites down and like a day

later and spend some time and if the

database back up there

so anyway that is that's all that I had

in this presentation but there's there's

tons of great information I want to give

a special shout-out to chromatic a

little about who like a lot of times

when I'm stuck on something in Jenkins

I'm like searching on Google and it's

like Oh chromatic has an article on that

Oh chromatic has an article on this so

there are definitely a lot of Drupal

companies using a lot of Jenkins and a

lot of a lot of the companies do a great

job of like showing here's the best way

to do this here's how we did this and

another thing that I didn't touch on

much is Drupal 8 specific stuff because

this particular site that I was demoing

is a Drupal 7 site but what chromatic

has some good information on how they do

deploys and backups and things Phil

Norton also has an old presentation from

I said rupal or Jenkins hasn't really

changed since 2004 terms of the UI so

once once blue ocean comes out

everything you've learned about the way

that you click around in Jenkins becomes

worthless but you know that's that's

life in software though so that is all

that I have I'm going to switch back to

the demo over here and let it run but

questions in discussion

and Jenkins has a huge system of plugins

that that get lab still doesn't match

perfectly the other thing that that

seems to be yeah the other thing that I

don't like all the time about get lab

and get lab CI in particular the

resource usage has been worse than

Jenkins sometimes for me and a lot of

some of the things that they use are

really complicated they're very cool but

they're really complicated and Jenkins

it's like it's literally a Java blob of

Java and it spits out a very detailed

log and when things go wrong it's pretty

easy to figure out that went wrong 99%

of the time it's out of memory with get

labs sometimes I've hit issues that took

longer to debug it's getting a lot

better the last time I really who's

getting Ivan earnest was about a year

ago and things have gotten a lot better

but if you're completely new to it at

all

I wouldn't say definitely is Jenkins I'd

say look at a couple options use the

quick start like run the Jenkins docker

container around the get lab docker

container see which one really fits with

your model of how things should work

because it should be pretty easy to get

the first couple things going you know

if it if it takes you six hours to get

cron running through your CI tool you

might not want to use that CIT

Drupal you know which for a lot of

people it does then is it really so

we're working on that though there's a

core issue for oh this finished so at

the end it installed Drupal here and

then it runs a test Drupal context and

in order to prove that the head is

working correctly as a developer I need

to run a simple interface test it says

given I'm on the home page then I should

see a no front page content it's been

created yet that's about it you know

very simple but if you had a suite of

tests you could have different parts of

it run by tags or you can run them all

you can also what a lot of people do is

they have like they're essential to us

like every time we commit we want to run

ten or twenty tests that make sure

little things are working but when we

before we go to prod we want to make

sure everything works and it's like a

three hour test it you know sometimes

for some sites you do have lots of tests

around and things like that and they

take forever so you can split that out

and have two jobs one that is like for

every commit and then one that's like

I'm deploying the prod do everything for

me and yet the other cool thing with

Jenkins that get loud I think kind of

has this now ansible and has a 2x which

is similar but a little different a lot

of the tools are getting up to speed on

it but Jenkins is pretty good about like

if you have a QA team that they should

only have access to QA environments and

QA jobs and if we had like a Java team

that should only do Java things and a

PHP team that does Drupal things you can

split that access up pretty easy making

these jenkins folders and jenkins tabs

in the interface to sort things a little

better so that is one thing that I like

about Jenkins if you have a large team

of people

it's it's pretty good for that case it's

almost so what their sample does it have

those building figs in it just for like

reference yeah that's another thing I

completely forgot to mention so as I

said if you're not if you're developing

in production that's not I don't like

doing that ever so what I do and since

like your entire jenkins history and

everything is in the Jenkins home folder

to to get those in your codebase all you

have to do is create a jobs folder in

your Jenkins codebase and stick all the

config files so this is the config file

for the b-hat run test that I just ran

you can see the script is in here

they're just their XML files

that's what Java EE earned that's what

Jenkins uses and you can see with all

the things that it's doing so I just I

log in to the server I copy the config

file out of out of that job and put it

in a code base and then in Hansel

there's there is it so there's there's

an ansible task right here that

literally just copies that jobs folder

to the Jenkins home jobs folder and

that's it and you can have structures

and folders in there it doesn't matter

one cool thing too is with the way that

an Sable's copy module works if you have

like four jobs under version control and

somebody had to create an emergency job

on Jenkins and you push an update they

won't delete that job because it's not

in your local code base it'll leave that

there it's also kind of a curse because

if somebody's going wrong and adding

jobs in your life Jenkins server you

have to like sculpt them and delete it

yourself but that's how that's set up so

all these jobs are in the codebase you

guys can access them I'm going to be

making them like I think right now I

have now this is hard-coded the list

yeah this list is hard-coded so if you

tried running this on your site inside

would be I don't have Jeff curing that

kind of that's stupid

we have group of VM running in prod for

our accountable commerce demos and we

have a multi-site for future payment

gateway and this I have a hack job like

circle CI setup but I want more control

backups and yeah this is yeah I mean

like I said this is literally what I'm

moving all my stuff to because I have an

old Jenkins server but it was that was

when I was first learning Jenkins and

it's it was one of where is that that

was one of these servers it is one of

these servers I haven't unpacked but and

I keep it updated but it's just not in

good shape so but yeah I'm gonna make

this this will be a template instead of

a just straight-up XML file and the list

of servers will be a configuration

variable thing with jinkins if you're

using docker you can add a deacons

container

yeah

and in the way that I have this set up

is it uses a docker it uses a docker

container as the base so if I go to dr.

compose it just grabs it grabs a copy as

of 1 to 16

I actually just add an 1804 yesterday so

probably works on there I haven't tested

the Jenkins role yet but anyway it grabs

the bunt to 16 with ansible inside it

already so that you can run this

playbook inside the docker container and

set up all the services and it has like

a mock systemd set up in there so you

can run it as if it were a live server

one thing that that does requires that

you use the privileged option in docker

which some people are a little nervous

about but as long as you don't poke them

access to your computer to the outside

it's not I don't feel it's a big deal

any other questions yeah what was the

login shows in console the timestamps

I think it's called timestamp er it's in

let me look at the config list here

timestamp er yeah and so that this setup

also the way that might Jenkins role

works is you give it a list of plugins

and it will install them all for you so

you don't have to go in that list and

check a bunch of boxes and click install

so some of the ones that I always always

always use is get because everything I

do is with get an see color adds colors

to the console otherwise it's always

black and white and I really hate black

and white I like color although dr.

strangelove was black and white so so

you sort of I to name this presentation

roll strategy is the one that gives you

the ability to have good permissions

broken out by different roles

parameterised schedule or lets you have

parameters in your schedules so that you

can have multiple schedules on one job

that do different things

timestamp Brett's time stamps WS cleanup

is workspace clean up and that makes it

easy like a lot of people in look like

writing a thing that says like RM

darkish are everything inside this

folder but a that's dangerous because

you could have the wrong path like what

if you CD it into a different folder

they deleted something you want to and B

if something in your job fails and it

doesn't get to that point then you know

here you're out of luck Jenkins also has

the ability I think I have it in here

you can run a task after the build a

post build tasks whoops

a post build task and you can say it

uses regular expressions to say like

when should I run so in this case I

always wanted to run I have done dot

masterís which means no matter what the

output says always do this when I

haven't removed the docker container

because even if my job fails I wanted to

remove that docker container so I don't

have like a thousand docker containers

which I accidentally did a couple years

ago and then that filled up the disk and

the RAM and the CPU Jenkins server died

and it was like not running jobs for a

day and then anyway so now I always make

sure I removed our containers after the

jobs over job that starts failing that

runs and you know that lots of docker

containers

why is the CPU my CPU is still going

insane

at least was something in doctor

yeah Dockers doing something like

anyways any other questions

what time is it we're good on time I

think all right well thanks everybody

[Applause]