December is for many people a period of reflection or thought. So I decided to reflect upon last year’s things and thoughts — each day until Christmas. This is day 8.
Continue reading “X-Mas Musings #8 – First Open-Source Project”
A while ago I described how to get release notes from JIRA with Groovy, so I thought it’s time to do a small follow up which is long overdue :-).
Let’s split things up to manageable classes with their own responsibility:
We’ll start at the bottom, so we’ll build upon the classes we already have.
Although a lot of the date we’re getting back from JIRA can be used as-is in Groovy, I decided to make the status of a JIRA issue, e.g. “Open”, into a trivial enum.
enum JiraStatus {
OPEN("Open"), IN_PROGRESS("In Progress"), RESOLVED("Resolved"), REOPENED("Reopened"), CLOSED("Closed");
JiraStatus(String value) { this.value = value }
private final String value
String value() { return value }
}
I’ll stick to Strings in the remainder of this post 🙂
Queries you can execute within JIRA itself in the Issue Navigator can also be performed through the REST API. Because there are several elements which constitute a JQL-query I decided to use the builder pattern to programmatically create queries through the JiraQueryBuilder.
import groovyx.net.http.RESTClient
import JiraStatus
/**
* Builder for a JQL-snippet, used for the JIRA search REST API.
*/
class JiraQueryBuilder {
private String project
private int[] sprintIds
private String sprintFunction
private String[] issueKeys
private String[] issueTypes
private String[] excludedIssueTypes
private String[] resolution
private JiraStatus[] status
private String[] labels
private String fixVersion
private String orderBy
/**
* Create a new query builder, querying by default everything for specified project, ordered by key.
*/
public JiraQueryBuilder(String project) {
assert project
this.project = project
withIssueTypes("standardIssueTypes()")
withOrderBy("key")
}
JiraQueryBuilder withSprintIds(int...sprintIds) {
this.sprintIds = sprintIds
return this
}
JiraQueryBuilder withIssueKeys(String...issueKeys) {
this.issueKeys = issueKeys
return this
}
JiraQueryBuilder withIssueTypes(String... issueTypes) {
this.issueTypes = issueTypes
return this
}
JiraQueryBuilder withExcludedIssueTypes(String... excludedIssueTypes) {
this.excludedIssueTypes = excludedIssueTypes
return this
}
JiraQueryBuilder withResolution(String... resolution) {
this.resolution = resolution
return this
}
JiraQueryBuilder withStatus(JiraStatus... status) {
this.status = status
return this
}
JiraQueryBuilder withLabels(String... labels) {
this.labels = labels
return this
}
JiraQueryBuilder withFixVersion(String fixVersion) {
this.fixVersion = fixVersion
return this
}
JiraQueryBuilder withOrderBy(String orderBy) {
this.orderBy = orderBy
return this
}
String build() {
String jql = "project = ${project}"
if (sprintIds) {
jql += " AND sprint in (${sprintIds.join(',')})"
}
if (sprintFunction) {
jql += " AND sprint in ${sprintFunction}"
}
if (issueKeys) {
jql += " AND issuekey in (${issueKeys.join(',')})"
}
if (issueTypes) {
jql += " AND issuetype in (${issueTypes.join(',')})"
}
if (excludedIssueTypes) {
jql += " AND issuetype not in (${excludedIssueTypes.join(',')})"
}
if (status) {
// turn (IN_PROGRESS, RESOLVED) into (Resolved, In Progress)
jql += " AND status in (${status*.value().join(',')})"
}
if (resolution) {
jql += " AND resolution in (${resolution.join(',')})"
}
if (labels) {
jql += " AND labels in (${labels.join(',')})"
}
if (fixVersion) {
jql += " AND fixVersion = \"${fixVersion}\""
}
if (orderBy) {
jql += " order by ${orderBy}"
}
return jql
}
}
Finally I needed to encapsulate the connection to a JIRA instance, allowing me to (HTTP) GET and POST from and to it, including using the proper credentials.
// require(groupId:'org.codehaus.groovy.modules.http-builder', artifactId:'http-builder', version:'0.5.2')
import net.sf.json.groovy.*
import groovyx.net.http.RESTClient
import static JiraStatus.*
/**
* JIRA REST client wrapper around groovyx.net.http.RESTClient
*/
class JiraRESTClient extends groovyx.net.http.RESTClient {
private static final String DEFAULT_PROJECT = "XXX"
private static final String DEFAULT_SEARCH_URL = "http://jira/rest/api/latest/"
String username
String password
def credentials = [:]
/**
* Create a REST client using Maven properties jira.username and jira.password for JIRA username and password.
*/
static JiraRESTClient create(def project) {
String jiraUsername = project.properties['jira.username']
String jiraPassword = project.properties['jira.password']
if (!jiraUsername?.trim()) {
throw new IllegalArgumentException("Empty property: jira.username")
}
if (!jiraPassword?.trim()) {
throw new IllegalArgumentException("Empty property: jira.password")
}
return create(jiraUsername, jiraPassword)
}
/**
* Create a REST client using provided JIRA username and password.
*/
static JiraRESTClient create(String username, String password) {
return new JiraRESTClient(DEFAULT_SEARCH_URL, username, password)
}
private JiraRESTClient(String url, String username, String password) {
super(url)
assert username
assert password
log.debug "Created for user=${username}, url=" + url
this.username = username;
this.password = password;
credentials['os_username'] = this.username
credentials['os_password'] = this.password
}
def get(String path, def query) {
try {
def response = get(path: path, contentType: "application/json", query: query)
assert response.status == 200
assert (response.data instanceof net.sf.json.JSON)
return response
} catch (groovyx.net.http.HttpResponseException e) {
if (e.response.status == 400) {
// HTTP 400: Bad Request, JIRA returned error
throw new IllegalArgumentException("JIRA query failed, response data=${e.response.data}", e)
} else {
throw new IOException("JIRA connection failed, got HTTP status ${e.response.status}, response data=${e.response.data}", e)
}
}
}
def post(String path, def body, def query) {
try {
def response = post(path: path, contentType: "application/json", body: body, query: query)
return response
} catch (groovyx.net.http.HttpResponseException e) {
if (e.response.status == 400) {
// HTTP 400: Bad Request, JIRA returned error
throw new IllegalArgumentException("JIRA query failed, got HTTP status 400, response data=${e.response.data}", e)
} else {
throw new IOException("JIRA connection failed, got HTTP status ${e.response.status}, response data=${e.response.data}", e)
}
}
}
/**
* Search JIRA with provided JQL e.g. "project = XXX AND ..."
* @returns response
* @throws IllegalArgumentException in case of bad JQL query
* @throws IOException in case of JIRA connection failure
* @see JiraQueryBuilder to create these JQL queries
*/
def search(String jql) {
assert jql
def query = [:]
query << credentials
query['jql'] = jql
query['startAt'] = 0
query['maxResults'] = 1000
log.debug "Searching with JQL: " + jql
return get("search", query)
}
Credentials are appended in plain sight to the url, but for internal use I found this acceptable. The search() gets a JQL string passed, appends the credentials and delegates to our own get() which actually performs groovyx.net.http.RESTClient‘s get() to make it happen.
The query to get all Resolved or Closed issues for a certain fix version can be encapsulated in a getReleaseNotes method, such as seen below.
def getReleaseNotes(String fixVersion) {
String jql = new JiraQueryBuilder(DEFAULT_PROJECT)
.withStatus(RESOLVED, CLOSED)
.withFixVersion(fixVersion)
.build()
return search(jql)
}
I’m still kickstarting this script with e.g. the gmaven-plugin which allows me to get my JIRA credentials from e.g. settings.xml, while the fix version for the JIRA query can be supplied as a property on the command-line.
JIRA is Atlassian‘s kick-ass issue tracker and in use in many projects I’ve been in. In my current project we use JIRA’s release notes after a certain version has been released, but some departments need the release notes fixed inside the created Maven artefact(s). For this we copy the release notes from JIRA into ReleaseNotes.xml to produce a PDF with the docbkx-maven-plugin – a tedious, manual process.
So, I set out to automate a few things: get the release notes from JIRA and convert to DocBook – all programmatically. Continue reading “Release Notes with Groovy and JIRA REST API”
Knowing which version of your application has been deployed on a certain environment is essential for today’s software development and maintenance. Have you ever needed to support a customer which reported a bug in a feature for which you already provided a patch in version control? That you asked yourself why the heck that fix isn’t on production and you don’t even know where or how to verify this?
There are many ways of versioning (which by itself is a very broad concept) your software but a simple mechanism to track what’s being deployed could be;
You don’t create builds from your local workspace these days, do you? 🙂
Below a simple outline of achieving this with Jenkins, creating a Mavenized webapplication (WAR), passing along a build number and reading it back in application code e.g. a JSF backing bean.
Define a property in your Maven pom.xml which holds the build number which we can overridde externally later on:
<properties> <build.number>SNAPSHOT</build.number> </properties>
If your application is packaged as a WAR (<packaging>war</packaging>) you can instruct the Maven War plugin to create a manifest file. This is a META-INF/MANIFEST.MF file which we can pass along some meta information about a JAR or in this case, the WAR.
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<manifest>
<addDefaultImplementationEntries>true</addDefaultImplementationEntries>
</manifest>
<archive>
<manifestEntries>
<Specification-Title>${project.name}</Specification-Title>
<Specification-Version>${project.version}</Specification-Version>
<Implementation-Version>${build.number}</Implementation-Version>
</manifestEntries>
</archive>
</configuration>
</plugin>
</plugins>
</build>
You could leave out some stuff, but the essential part is Implementation-Version which uses the ${build.number} property. This will create a MANIFEST.MF which sort of looks like the following:
Manifest-Version: 1.0 Implementation-Version: SNAPSHOT Built-By: jenkins Build-Jdk: 1.6.0_31 Specification-Title: Example Project Name Created-By: Apache Maven Specification-Version: 0.0.1 Archiver-Version: Plexus Archiver
If you don’t overridde build.number it’ll default to SNAPSHOT. Well, for now this doens’t add much value, but now we can have Jenkins set it for us too.
If you would translate the above mvn commandline to a Jenkins Maven job you can use several of Jenkins’ environment variables to pass along in properties. E.g.
You can take anything from the list (if applicable, such as SVN_REVISION when using Subversion) and pass it along as property. Let’s take BUILD_NUMBER as our build.number property: mvn clean install -Dbuild.number=${BUILD_NUMBER}
In Jenkins it would look like
If you would run the Jenkins job again, its current build bumber would be used and end up in the manifest file.
Manifest-Version: 1.0 Implementation-Version: 153 Built-By: jenkins
To wrap up, you obviously need to make this version visible in a running application. You could add the version to the HTML header in comments or display it on some sort of health page. For working with manifest files you can read the Java Manifest Files Tutorial. To read from it use java.util.jar.Manifest and java.util.jar.Attributes as you can see in the (JSF) example below.
ExternalContext application = FacesContext.getCurrentInstance().getExternalContext();
InputStream inputStream = application.getResourceAsStream("/META-INF/MANIFEST.MF");
Manifest manifest = new Manifest(inputStream);
Attributes attributes = manifest.getMainAttributes();
String version = attributes.getValue("Implementation-Version");
There’s obviously more to it than meets the eye…or your requirements. You can now see which build of the application is running on a environment, and going back to your CI server – which generated the build number – and relate it to the code inside the build. We haven’t used Maven’s own project version, release plugins or other fancy tools yet, but if you start simple you can always expand into more elaborate schemes to suit your needs.
As part of evaluating an open-source framework which uses Google Web Toolkit I ran into some of the familiar issues I already had a few times earlier with the GWT (GWT 2.4 or 2.5) Maven plugin compilation running out of memory. You know, when you see…
…at the start:
[INFO] # [INFO] # There is insufficient memory for the Java Runtime Environment to continue. [INFO] # Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocate
…or during compilation:
[INFO] Compiling 12 permutations [INFO] Compiling permutation 1... [INFO] Process output [INFO] Error occurred during initialization of VM [INFO] Could not reserve enough space for object heap
This post is basically a compilation of the answers found online, and a quick reminder for myself and others who are running into similar problems with the fantastic GWT compiler :-).
1. First of all, check if you actually need to compile everything! Skip some browsers or locales – saving all kinds of permutations – if you don’t need it for development. Use the tips about how to speed up the GWT compiler and GWT compilation performance. There’s also an existing FAQ about GWT Debugging and Compiling.If you have control of your machine, cut down on other running applications and processes while you’re compiling.
2. Pass along some more memory
…to Maven, e.g.:
MAVEN_OPTS=-Xms1024m -Xmx1024m -XX:MaxPermSize=256m
…and/or to the gwt-maven-plugin, e.g.:
<configuration> <extraJvmArgs>-Xms768m -Xmx768m</extraJvmArgs> </configuration>
You’ll have to play a bit with the values, depending on the machine you’re working on and the actually available working RAM.
Add some permgen space, which is used when GWT compiles Java into Javascript. E.g.
-XX:MaxPermSize=1024m3. Try lowering the number of workers to a number explicitly below your available cores/cpu’s. If this number isn’t defined explicitly, the gwt-maven-plugin will set these to the number of available cores/cpu’s. Less workers = less parallellisation = less memory overhead.
<configuration>
<extraJvmArgs>-Xms768m -Xmx768m</extraJvmArgs>
<localWorkers>3</localWorkers>
</configuration>
4. Try having the “workers” work in threads, instead of spawning new processes.
<configuration>
<extraJvmArgs>-Xms768m -Xmx768m -Dgwt.jjs.permutationWorkerFactory=com.google.gwt.dev.ThreadedPermutationWorkerFactory</extraJvmArgs>
<localWorkers>3</localWorkers>
</configuration>
Hopefully this will save someone some valuable time finding solutions. If you have other options, feel free to add them in the comments.
I want to share something a software developer or operations guy every now and then encounters. And which I did yesterday.
It all started with our nightly Maven build failing last Saturday Jan 28 on a
java.lang.NoClassDefFoundError: javax/xml/bind/ValidationEventLocator
