Kotlin with Maven

Java, kotlin, maven, programming

Stand-alone Programs

Instructions and template for a stand-alone Java program built with Kotlin source code.

An easy way to start with is, since JetBrains is behind this whole thing, just install and run their IntelliJ IDEA and create a new project from IntelliJ with the language set to Kotlin and build system set to Maven. Check the option to include a sample code. It should then flesh out a template project.

In the generated pom.xml file, make these modifications:

  • Add a property main.class that defines the main entry class to the generated MainKt class (so you don’t have to copy and paste into places).
  • Modify the exec-maven-plugin‘s mainClass to be ${main.class}.
  • Add the maven-assembly-plugin so that a stand-alone JAR will be created to be run.
  • Optional: Add the maven-compiler-plugin if you have Java code alongside Kotlin code.
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xmlns="http://maven.apache.org/POM/4.0.0"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    ...
    <properties>
        ...
        <main.class>MainKt</main.class>
    </properties>

    ...

    <build>
        <sourceDirectory>src/main/kotlin</sourceDirectory>
        <testSourceDirectory>src/test/kotlin</testSourceDirectory>
        <plugins>
            <plugin>
                <groupId>org.codehaus.mojo</groupId>
                <artifactId>exec-maven-plugin</artifactId>
                <version>1.6.0</version>
                <configuration>
                    <mainClass>${main.class}</mainClass>
                </configuration>
            </plugin>
            ...
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.5.1</version>
                <executions>
                    <!-- Replacing default-compile as it is treated specially by maven -->
                    <execution>
                        <id>default-compile</id>
                        <phase>none</phase>
                    </execution>
                    <!-- Replacing default-testCompile as it is treated specially by maven -->
                    <execution>
                        <id>default-testCompile</id>
                        <phase>none</phase>
                    </execution>
                    <execution>
                        <id>java-compile</id>
                        <phase>compile</phase>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                    </execution>
                    <execution>
                        <id>java-test-compile</id>
                        <phase>test-compile</phase>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-assembly-plugin</artifactId>
                <version>3.4.2</version>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals> <goal>single</goal> </goals>
                        <configuration>
                            <archive>
                                <manifest>
                                    <mainClass>${main.class}</mainClass>
                                </manifest>
                            </archive>
                            <descriptorRefs>
                                <descriptorRef>jar-with-dependencies</descriptorRef>
                            </descriptorRefs>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

Building

Just running mvn install should build the stand-alone JAR:

  • target/<your project name>-1.0-SNAPSHOT-jar-with-dependencies.jar

Running

As a stand-alone JAR

To run the stand-alone JAR as a Java program:

java -jar target/<project name>-1.0-SNAPSHOT-jar-with-dependencies.jar ...

where ... are any arguments (args) you want to pass to the program.

With Maven

Because by default IntelliJ added the plugin exec-maven-plugin, the program can also be run without having to build the JAR. This is useful during development before deployment:

mvn exec:java -Dexec.args="..."

Notice that passing arguments to the program is a bit cumbersome due to the way Maven works (using the -Dexec.args property).

Deployment

The easiest way to deploy is to copy that target/<project name>-1.0-SNAPSHOT-jar-with-dependencies.jar to where you want to deploy the program. Renaming the JAR file to be something shorter helps. Also: changing the version in the POM file to a production SemVer value will get rid of that “SNAPSHOT” suffix.

Some Cloud Storage Services

Java, programming

Azure Blob Storage (“Azure”) and Google Cloud Storage (“GCS”) are now supported as well as AWS S3 (“S3”) by log4j-s3-search project.

While working on adding the two options, I learned a bit about the three storage services. These services’ similarity to one another may not be a coincidence. After all, they are competing offerings from competing service providers, so months/years of competitive analyses may just yield a bunch of similar things.

HTTP and Language SDKs

All three services have the basic HTTP interface:

However, all three services also have language bindings, or client SDKs, for popular programming languages (e.g. Java, Python, C, etc.). And my experience is that working with these SDKs is definitely easier than dealing with them on your own via HTTP. This is especially true for AWS S3, considering the logic used to sign a request.

Storage Model

The models are similar for all three:

  • A global namespace is used for a container of blobs. This can be referred to either a container (Azure) or a bucket (S3 and GCS).
  • Within each container are key-value pairs where the key is a string of characters that may resemble a path-like format to mimic that used by the folder hierarchy in file systems (e.g. “/documents/2020/April/abc.doc“).
    The consoles for these services may interpret keys of that format and present a hierarchical tree-like interface to further the illusion of the hierarchy. However, keep in mind that underneath in the implementation, a key is just a string of characters.
  • The value is a binary stream (a “blob”) of arbitrary data. Typically the services will allow attaching metadata to the key-value entry. One of the common metadata properties is “Content-Type” that is similar to the HTTP header of the same name in usage: to hint to the users of the blob what the content is (e.g. “text/plain,” “application/json,” “application/gzip,” etc.).

Not-So-Quick Walkthrough

The following steps are what I went through in order to upload a file into the services.

In order to use these services, of course, an account with the service is required. All three services have a “free” period for first-time users.

Set Up Authentication

S3

Sign into https://console.aws.amazon.com/

Create an access key. Despite the name, creating one actually yields two values: an access key and a corresponding secret key. The process is a bit clumsy. Be sure to write down and/or download and save the secret key because it is only available during this time. The access key is listed in the console. If the secret key is lost, a new access key needs to be created.

Create a subdirectory .aws and a file credentials under your user’s home directory ($HOME in Linux/Mac, %USERPROFILE% in Windows):

.aws/
    credentials

The contents of the file should be something like (substitute in your actual access and secret keys, of course):

[default]
aws_access_key_id = ABCDEFGABCDEFGABCDEFG
aws_secret_access_key = EFGABCDEFG/ABCDEFGABCDEFGAABCDEFGABCDEFG

That should be sufficient for development purposes.

Azure

Sign into https://portal.azure.com/

Create a Storage account. One of the Settings for the Storage account is Access keys. A pair of keys should have been generated. Any of them will work fine. Just copy down the Connection string of a key to use.

The connection string will be used to authenticate when using the SDK.

Optional: one common pattern I see is that an environment variable AZURE_STORAGE_CONNECTION_STRING is created whose value is the connection string. Then the code will simply look up the environment variable for the value. This will avoid having to hard-code the connection string into the source code.

GCS

Sign into https://console.cloud.google.com/

Create a project. Then create a Service account within the project.

In the project’s IAM > Permissions page, add the appropriate “Storage *” roles to the service account.

Add “Storage Admin” to include everything. After a while, the “Over granted permissions” column will have information on the actual permissions needed based on your code’s usage, and you can adjust then.

Then create a key for the Service account. The recommended type is JSON. This will download a JSON file that will be needed.

Set the environment variable GOOGLE_APPLICATION_CREDENTIALS to the full path to where the JSON file is stored.

Write Code to Upload File

The examples below are in Java.

S3

String bucketName = "mybucket";
String key = "myfile";
File file = new File(...);  // file to upload

AmazonS3Client s3 = (AmazonS3Client)AmazonS3ClientBuilder
    .standard()
    .build();
if (!client.doesBucketExist(bucketName)) {
    client.createBucket(bucketName);
}
PutObjectRequest por = new PutObjectRequest(
    bucketName, key, file);
PutObjectResult result = client.putObject(por);

Azure

This is using the v8 (Legacy) API that I ended up doing. To do this with the newer v12 API, see https://docs.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-java#upload-blobs-to-a-container

String containerName = "mycontainer";
String key = "myfile";
File file = new File(...);  // file to upload

String connectionString = System.getenv(
    "AZURE_STORAGE_CONNECTION_STRING");
CloudStorageAccount account = CloudStorageAccount.parse(
    connectionString);
CloudBlobClient blobClient = account.createCloudBlobClient();
CloudBlobContainer container = blobClient.getContainerReference(
    containerName);
boolean created = container.createIfNotExists(
    BlobContainerPublicAccessType.CONTAINER, 
    new BlobRequestOptions(), new OperationContext());

CloudBlockBlob blob = container.getBlockBlobReference(key);
blob.uploadFromFile(file.getAbsolutePath());

GCS

While the other two services have convenient methods to upload a file, the GCS Java SDK does not; it only has a version that uploads a byte[] which is dangerous if your data can be large.

Internet to the rescue, I guess, since this article has one solution by implementing our own buffering uploader:

private void uploadToStorage(
    Storage storage, File uploadFrom, BlobInfo blobInfo) 
    throws IOException {
    // Based on: https://stackoverflow.com/questions/53628684/how-to-upload-a-large-file-into-gcp-cloud-storage

    if (uploadFrom.length() < 1_000_000) {
        storage.create(
            blobInfo, Files.readAllBytes(uploadFrom.toPath()));
    } else {
        try (WriteChannel writer = storage.writer(blobInfo)) {
            byte[] buffer = new byte[10_240];
            try (InputStream input = Files.newInputStream(uploadFrom.toPath())) {
                int limit;
                while ((limit = input.read(buffer)) >= 0) {
                    writer.write(
                        ByteBuffer.wrap(buffer, 0, limit));
                }
            }
        }
    }
}

With that defined, then the upload code is:

String bucketName= "mybucket";
String key = "myfile";
File file = new File(...);  // file to upload

Storage storage = StorageOptions.getDefaultInstance()
    .getService();
Bucket bucket = storage.get(bucketName);
if (null == bucket) {
    bucket = storage.create(BucketInfo.of(bucketName));
}

BlobId blobId = BlobId.of(bucketName, key);
BlobInfo blobInfo = BlobInfo.newBuilder(blobId).build();
uploadToStorage(storage, file, blobInfo);

Publishing a Java project into MVN Repository

Java, maven, programming

It turns out that this is quite an involved process with a lot of configuration and coordination w/ Sonatype. It took several days, upwards to a week or so, for me to get this working. Plan accordingly.

These notes are based on various articles out there, but updated to reflect learnings I had when going through the process.

Qualifications:

Here are some specifics for my project.

  • Project is hosted in GitHub as a public repo.
  • Project is a Maven project with a POM file at the root. It’s a multi-module project, but that shouldn’t change anything other than the fact that I publish the modules individually. There may be a way to publish all the modules simultaneously, but I haven’t explored that option.

Prerequisite:

  • Create an account with Sonatype. Start at https://issues.sonatype.org/secure/Dashboard.jspa and click “Sign up.”
  • Create an issue for the new project:
    • Project: “Community Support – Open Source Project Repository Hosting”
    • Type: “New Project”
    • Group Id: a domain I own (or something based on the project hosting like io.github.<user narme> or com.github.<user name>.
    • Project URL: GitHub URL to the project.
    • SCM URL: path to the repo (https://github.com/…/<project>.git)
    • Username(s): the Sonatype user name.

Configure project with Maven src and Javadoc plugins:

See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L106-L134

Create and publish an RSA key

  • Create a RSA key (e.g. using GnuPG’s “gpg –full-gen-key with 2048 bits)
  • Publish the key (e.g. “gpg –keyserver pool.sks-keyservers.net –send-key <my RSA key ID>“)

Configure Maven to talk to Sonatype servers:

Create/edit the file settings.xml under M2_HOME/config or M2_REPO/config:

Find the servers section and add:

<server>
  <id>ossrh</id>
  <username>my_sonatype_username</username>
  <password>my_sonatype_password</password>
</server>

Find the profiles section and add:

<profile>
  <id>ossrh</id>
  <activation>
    <activeByDefault>true</activeByDefault>
  </activation>
  <properties>
    <gpg.keyname>my RSA key ID</gpg.keyname>
    <gpg.passphrase>my RSA key's passphrase</gpg.passphrase>
  </properties>
</profile>

Use the same GPG key generated above.

Configure project for Maven DEPLOY Plugin:

Add the Maven Deploy Plugin. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L64-L76

Add a distributionManagement section to the project POM. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L137-L146

Add an scm section to the project POM. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L147-L152

Configure Project for Nexus-Staging-Maven Plugin:

See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L95-L105

If using a different server ID than “ossrh,” keep in sync with the entries defined in the distributionManagement section and also the configuration in the Maven conf/settings.xml.

Configure project for Maven Release Plugin:

Add the Maven Release Plugin. See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L77-L94

Configure Project to Sign artifacts when releasing:

See https://github.com/bluedenim/log4j-s3-search/blob/master/appender-core/pom.xml#L154-L180

Preparation of Release:

Finally, to prepare for a project to be released,

  • Build the project once with mvn install
  • Fix all the issues that come up with unit tests and Javadoc.
  • Commit and push all the changes to GitHub.
  • Modify the project’s version to be a “-SNAPSHOT” release for the release I want to make. For example, if I want to release a 1.0.0 version, use the version “1.0.0-SNAPSHOT” for the project. However, none of the project’s dependencies can be to SNAPSHOT versions.
  • Commit the change. No need to push to GitHub.

Run mvn release:prepare

The process will ask some questions. The release will be “1.0.0” in this example. The next release is probably “1.0.1-SNAPSHOT” as suggested, but this can always be modified later as needed.

Release TO SONATYPE:

Run mvn release:perform

This takes a while, and it will actually push artifacts to Sonatype’s servers. Things can go wrong here that range from intermittent network errors to errors with the setup of the repo.

Artifacts can be verified by logging into https://oss.sonatype.org/ with the Sonatype account created earlier and searching for the released artifacts.

Issues may need to be filed with Sonatype if the repo is set up incompletely. Even when things work correctly, it may take some time for things to propagate through the various servers before the artifacts show up in mvnrepository.com.

Once this works the first time, subsequent releases are more stable.

 

Addendum:

Parent POMs

To release parent POMs w/o triggering releasing the modules under the parent, add:

 -N -Darguments=-N

as documented here: http://maven.apache.org/maven-release/maven-release-plugin/faq.html#nonrecursive

 

 

SampleServlet: Java web app starter

Java, jetty, maven, programming

Prerequisite

Maven 3.x installed – try http://maven.apache.org/download.cgi
Important: earlier versions of Maven, such as 2.x, won’t work correctly

Set-up

Locate a subdirectory and run:

mvn archetype:generate

When prompted, use maven-archetype-quickstart as the archetype to use.  Enter the groupId, artifactId, version, etc.

Add dependencies and plugins into pom.xml

A relatively useless pom.xml file is generated.  Modify it to include some dependencies and plugins:

<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.van</groupId>
<artifactId>sampleweb</artifactId>
<packaging>jar</packaging>
<version>1.0</version>
<name>sampleweb</name>
<url>http://maven.apache.org</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.5</version>
</dependency>
</dependencies>

<build>
<finalName>sampleweb</finalName>
<plugins>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.6</source>
<target>1.6</target>
</configuration>
</plugin>
<plugin>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-maven-plugin</artifactId>
<configuration>
<contextPath>/</contextPath>
</configuration>
</plugin>

</plugins>
</build>
</project>

Add web.xml per Java Servlet specs

Create the subdirectory src/main/webapp/WEB-INF/ and add into it <strong?web.xml:

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd"
version="2.5">
<servlet>
<servlet-name>SampleServlet</servlet-name>
<servlet-class>org.van.SampleServlet</servlet-class>
</servlet>

<servlet-mapping>
<servlet-name>SampleServlet</servlet-name>
<url-pattern>/SampleServlet/*</url-pattern>
</servlet-mapping>
</web-app>

Create SampleServlet.java

Under src/main/java/, delete the App.java generated and add the servlet file under the appropriate package subdirectory (e.g. “org/van/SampleServlet.java”):

package org.van;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

public class SampleServlet extends HttpServlet {
/*
Redirects to some other page. Sample usage:
http://localhost:8080/SampleServlet?destination=http://www.therealvan.com
*/
public void doGet(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, java.io.IOException {

String dest = req.getParameter("destination");
resp.sendRedirect(dest);
}
}

Run

Just invoke the Jetty run command:

mvn jetty:run

Zip of starter project

Here is a zip of the starter project to extract wherever to get started:
sampleweb.zip

Details

The sample project uses the following:

  • Java Servlet 2.5 API
  • Maven compiler plugin
  • jetty-maven-plugin