Difference between revisions of "Setting up a Jenkins server"
(→Stage three) |
(→= NOTE: This documentation is out of date.) |
||
| (29 intermediate revisions by 4 users not shown) | |||
| Line 1: | Line 1: | ||
| + | === NOTE: This documentation is out of date. == | ||
| + | Current work on setting up a Jenkins server for the TEI take the approach of building a Docker container. See the TEI Jenkins repository [https://github.com/TEIC/Jenkins/ https://github.com/TEIC/Jenkins/] for detailed information. | ||
| + | --- | ||
| + | |||
This page contains instructions and a script for setting up a Jenkins Continuous Integration Server for automated building of all of the TEI products. | This page contains instructions and a script for setting up a Jenkins Continuous Integration Server for automated building of all of the TEI products. | ||
| − | |||
== What is a Jenkins server? == | == What is a Jenkins server? == | ||
| − | + | Jenkins is a continuous integration server, a tool from modern software engineering. Its role is to notice whenever a change is made to the work-in-progress text of the TEI standard and perform a range of tasks, including building building each of the language variants and a number of tests. Jenkins catches a broad range of technical issues and complains very loudly about them to the users it thinks caused them. | |
| + | When it detects a change (because someone has edited one of the source code files), it downloads the changes and starts building the codebase to create an updated version of the project, whatever that is. In the case of the TEI, there are seven different sub-projects, each of which is built separately: the OxGarage conversion server, the Roma schema-building tool, the TEI Stylesheets, an XSLT 1.0 version of the Stylesheets, TEI P5, TEI P5 Documentation (the Guidelines etc.), and some TEI P5 test cases. Some of these jobs depend on others -- for instance, all of the TEIP5-* projects are built using the stylesheets created from the Stylesheets project. | ||
== Why would I want to create a Jenkins server? == | == Why would I want to create a Jenkins server? == | ||
| Line 11: | Line 15: | ||
A Jenkins server is useful to project administrators because it will automatically create new builds whenever the codebase changes, so the admins don't have to remember to do that themselves. It will also notify you when a build fails, perhaps because a recent change to the source contained an error. Programmers working on the codebase can use this feature to check the changes they have made to make sure they're OK, and roll them back if they don't work, without having to have a complete build environment set up on their own computers. | A Jenkins server is useful to project administrators because it will automatically create new builds whenever the codebase changes, so the admins don't have to remember to do that themselves. It will also notify you when a build fails, perhaps because a recent change to the source contained an error. Programmers working on the codebase can use this feature to check the changes they have made to make sure they're OK, and roll them back if they don't work, without having to have a complete build environment set up on their own computers. | ||
| − | The TEI's main Jenkins server, | + | The TEI's main Jenkins server, at the University of Victoria, is at [http://jenkins.tei-c.org http://jenkins.tei-c.org/]. There you can see the multiple build targets and look at their status -- when they were last built, when the last build failure occurred, etc. The University of Oxford also has a Jenkins server running at [http://jenkins2.tei-c.org/ http://jenkins2.tei-c.org/]. There is a third backup server running at [http://jenkins-paderborn.tei-c.org/ http://jenkins-paderborn.tei-c.org/]. |
| + | |||
| + | You might want to set up your own Jenkins server for a number of reasons: | ||
*To add extra redundancy to the TEI project build process | *To add extra redundancy to the TEI project build process | ||
| Line 17: | Line 23: | ||
*To learn about Jenkins so that you can help administer one of the main servers | *To learn about Jenkins so that you can help administer one of the main servers | ||
| + | == How do I go about creating a Jenkins server? == | ||
| − | + | There are four stages: | |
| − | + | 1. Build a basic headless Ubuntu server, using the Ubuntu Precise 12.04 (Long-Term Support) edition. | |
| − | + | 2. Install an Oxygen licence in the right location (some of the TEI projects require Oxygen). | |
| − | + | 3. Run our customization script to install all of the TEI and Jenkins code. | |
| − | + | 4. Make some final tweaks to your setup (setting up security for Jenkins). | |
You will need to be comfortable with installing Linux and doing a couple of things at the command line. If you've never done this kind of thing before, this project is probably not for you. | You will need to be comfortable with installing Linux and doing a couple of things at the command line. If you've never done this kind of thing before, this project is probably not for you. | ||
| + | |||
| + | Please note that the script is long and quite complicated, and it depends upon a variety of external services which may perhaps not be functioning correctly at the time when you run it. As a result, the script may fail. This is not really disastrous; you can open it in a text editor to see what it does, and run each command manually until the setup is complete. The script is intended as much to be a thorough documentation of how the TEI Jenkins servers work, and what is required in order to set one up, as it is to be a functional build script (although it has been thoroughly tested, and it does work). | ||
== Stage one == | == Stage one == | ||
| − | Download the latest distribution of the Ubuntu | + | Download the latest distribution of the Ubuntu Precise (12.04) server ISO from the Ubuntu download site ([http://www.ubuntu.com/download/ubuntu/alternative-download http://www.ubuntu.com/download/ubuntu/alternative-download]). |
| − | |||
| − | |||
| + | Install the distribution on the machine you're going to use as your server (or on a virtual machine, if you don't have a piece of hardware to dedicate to the job). Choose "Internet server" when you're asked what type of computer you're setting up, during the install process. Then do all the updates to make sure your Precise installation is fully updated. | ||
== Stage two == | == Stage two == | ||
| − | + | Log into your server, and place a copy of a valid Oxygen license file, licensekey.txt, in /usr/share/oxygen/. The file should contain the nine lines of text of the Oxygen license key (located between the license key start and end markers). A valid Oxygen license is required in order to build some of the TEI projects. | |
| + | == Stage three == | ||
| − | + | Download our [https://github.com/TEIC/TEI/blob/dev/Documents/Editing/Jenkins/jenkins_builder_script_2012.sh Jenkins_builder_script.sh] script from the TEI GitHub repository. Get the script onto your server somehow. Make it executable, and run it as root. You'll see some basic explanatory information as it starts up. When the script completes, you should have a working Jenkins machine. The Jenkins GUI will be running on its default port of 8080, and if you connect to your server on that port, you should see it start to run the first of the seven TEI jobs. The first run takes a long time, because it has to download all the source code from GitHub, but subsequent builds triggered by changes should be much quicker. | |
| − | + | == Stage four == | |
| − | Next, set up some security for your Jenkins installation, by using the Jenkins management interface on port 8080 There are various ways to do this, using different authentication methods, but this is the simplest: | + | Next, you need to set up some security for your Jenkins installation, by using the Jenkins management interface on port 8080 There are various ways to do this, using different authentication methods, but this is the simplest: |
* Click on Manage Jenkins, then Configure System. | * Click on Manage Jenkins, then Configure System. | ||
| Line 54: | Line 63: | ||
* Create a new user (called, for instance, "tei"), and give it full privileges (check everything for that user). | * Create a new user (called, for instance, "tei"), and give it full privileges (check everything for that user). | ||
* For the Anonymous user, check only the Read boxes in Overall and Job. | * For the Anonymous user, check only the Read boxes in Overall and Job. | ||
| + | * Click on Save at the bottom of the page to save your changes. | ||
Now log in as the new user ("tei") and set a password for yourself. Now anyone can see the progress of jobs, but only the tei user can log in and make changes. | Now log in as the new user ("tei") and set a password for yourself. Now anyone can see the progress of jobs, but only the tei user can log in and make changes. | ||
| − | |||
== What does your script actually do? == | == What does your script actually do? == | ||
| − | + | Basically, the builder script adds some repositories and installs some software from them, including the Jenkins server software. Then it downloads some configuration files from the TEI SourceForge repository, tweaks them a bit, and restarts Jenkins. You can read the script itself in the [https://github.com/TEIC/TEI/blob/dev/Documents/Editing/Jenkins/jenkins_builder_script_2012.sh GitHub repository]. | |
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
| − | |||
Latest revision as of 01:42, 19 December 2016
Contents
= NOTE: This documentation is out of date.
Current work on setting up a Jenkins server for the TEI take the approach of building a Docker container. See the TEI Jenkins repository https://github.com/TEIC/Jenkins/ for detailed information. ---
This page contains instructions and a script for setting up a Jenkins Continuous Integration Server for automated building of all of the TEI products.
What is a Jenkins server?
Jenkins is a continuous integration server, a tool from modern software engineering. Its role is to notice whenever a change is made to the work-in-progress text of the TEI standard and perform a range of tasks, including building building each of the language variants and a number of tests. Jenkins catches a broad range of technical issues and complains very loudly about them to the users it thinks caused them.
When it detects a change (because someone has edited one of the source code files), it downloads the changes and starts building the codebase to create an updated version of the project, whatever that is. In the case of the TEI, there are seven different sub-projects, each of which is built separately: the OxGarage conversion server, the Roma schema-building tool, the TEI Stylesheets, an XSLT 1.0 version of the Stylesheets, TEI P5, TEI P5 Documentation (the Guidelines etc.), and some TEI P5 test cases. Some of these jobs depend on others -- for instance, all of the TEIP5-* projects are built using the stylesheets created from the Stylesheets project.
Why would I want to create a Jenkins server?
A Jenkins server is useful to project administrators because it will automatically create new builds whenever the codebase changes, so the admins don't have to remember to do that themselves. It will also notify you when a build fails, perhaps because a recent change to the source contained an error. Programmers working on the codebase can use this feature to check the changes they have made to make sure they're OK, and roll them back if they don't work, without having to have a complete build environment set up on their own computers.
The TEI's main Jenkins server, at the University of Victoria, is at http://jenkins.tei-c.org/. There you can see the multiple build targets and look at their status -- when they were last built, when the last build failure occurred, etc. The University of Oxford also has a Jenkins server running at http://jenkins2.tei-c.org/. There is a third backup server running at http://jenkins-paderborn.tei-c.org/.
You might want to set up your own Jenkins server for a number of reasons:
- To add extra redundancy to the TEI project build process
- To add your own build targets for specific purposes
- To learn about Jenkins so that you can help administer one of the main servers
How do I go about creating a Jenkins server?
There are four stages:
1. Build a basic headless Ubuntu server, using the Ubuntu Precise 12.04 (Long-Term Support) edition.
2. Install an Oxygen licence in the right location (some of the TEI projects require Oxygen).
3. Run our customization script to install all of the TEI and Jenkins code.
4. Make some final tweaks to your setup (setting up security for Jenkins).
You will need to be comfortable with installing Linux and doing a couple of things at the command line. If you've never done this kind of thing before, this project is probably not for you.
Please note that the script is long and quite complicated, and it depends upon a variety of external services which may perhaps not be functioning correctly at the time when you run it. As a result, the script may fail. This is not really disastrous; you can open it in a text editor to see what it does, and run each command manually until the setup is complete. The script is intended as much to be a thorough documentation of how the TEI Jenkins servers work, and what is required in order to set one up, as it is to be a functional build script (although it has been thoroughly tested, and it does work).
Stage one
Download the latest distribution of the Ubuntu Precise (12.04) server ISO from the Ubuntu download site (http://www.ubuntu.com/download/ubuntu/alternative-download).
Install the distribution on the machine you're going to use as your server (or on a virtual machine, if you don't have a piece of hardware to dedicate to the job). Choose "Internet server" when you're asked what type of computer you're setting up, during the install process. Then do all the updates to make sure your Precise installation is fully updated.
Stage two
Log into your server, and place a copy of a valid Oxygen license file, licensekey.txt, in /usr/share/oxygen/. The file should contain the nine lines of text of the Oxygen license key (located between the license key start and end markers). A valid Oxygen license is required in order to build some of the TEI projects.
Stage three
Download our Jenkins_builder_script.sh script from the TEI GitHub repository. Get the script onto your server somehow. Make it executable, and run it as root. You'll see some basic explanatory information as it starts up. When the script completes, you should have a working Jenkins machine. The Jenkins GUI will be running on its default port of 8080, and if you connect to your server on that port, you should see it start to run the first of the seven TEI jobs. The first run takes a long time, because it has to download all the source code from GitHub, but subsequent builds triggered by changes should be much quicker.
Stage four
Next, you need to set up some security for your Jenkins installation, by using the Jenkins management interface on port 8080 There are various ways to do this, using different authentication methods, but this is the simplest:
- Click on Manage Jenkins, then Configure System.
- Check "Enable security".
- Choose "Jenkins's own user database".
- Choose "Matrix-based security".
- Create a new user (called, for instance, "tei"), and give it full privileges (check everything for that user).
- For the Anonymous user, check only the Read boxes in Overall and Job.
- Click on Save at the bottom of the page to save your changes.
Now log in as the new user ("tei") and set a password for yourself. Now anyone can see the progress of jobs, but only the tei user can log in and make changes.
What does your script actually do?
Basically, the builder script adds some repositories and installs some software from them, including the Jenkins server software. Then it downloads some configuration files from the TEI SourceForge repository, tweaks them a bit, and restarts Jenkins. You can read the script itself in the GitHub repository.
