Java HTTP Request with Basic Auth

This is how you do a simple HTTP request with Java. These code performs the actual HTTP request and saves the response in a String variable.

URL url = new URL(address);
URLConnection conn = url.openConnection();
conn.setConnectTimeout(30000); // 30 seconds time out
String line = "";
StringBuffer sb = new StringBuffer();
BufferedReader input = new BufferedReader(new InputStreamReader(conn.getInputStream()) );
while((line = input.readLine())!=null)
  sb.append(line);
input.close();
String response = sb.toString();

If the HTTP server requires Baisc Auth this code will fail. To make it work for Basic Auth these 3 additional lines are required.

String user_pass = username + ":" + password;
String encoded = Base64.encodeBase64String( user_pass.getBytes() );
conn.setRequestProperty("Authorization", "Basic " + encoded);

The whole method looks like that:

public String getHttpResponse(String address, String username, String password) throws Exception {
  URL url = new URL(address);
  URLConnection conn = url.openConnection();
  conn.setConnectTimeout(30000); // 30 seconds time out

  if (username != null && password != null){
    String user_pass = username + ":" + password;
    String encoded = Base64.encodeBase64String( user_pass.getBytes() );
    conn.setRequestProperty("Authorization", "Basic " + encoded);
  }

  String line = "";
  StringBuffer sb = new StringBuffer();
  BufferedReader input = new BufferedReader(new InputStreamReader(conn.getInputStream()) );
  while((line = input.readLine()) != null)
    sb.append(line);
  input.close();
  return sb.toString();
}

How to get notified about out-dated dependencies in build.sbt?

Scala SBT is the build tool for the programming language Scala. You can specify your project dependencies in the “build.sbt” file, similar to the pom.xml file in Maven. A build.sbt file can look like this.

name := "s3crate"

organization := "codexica"

version := "0.0.1-SNAPSHOT"

scalaVersion := "2.10.2"

libraryDependencies += "net.java.dev.jets3t" % "jets3t" % "0.9.0"

libraryDependencies += "com.amazonaws" % "aws-java-sdk" % "1.3.33"

But how do you get notified about new versions of your dependencies?

VersionEye now supports Scala SBT. VersionEye can monitor your build.sbt file on GitHub / Bitbucket. If you are logged in to VersionEye you just have to navigate to your GitHub / Bitbucket repository and flip the switch beside the build.sbt file.

01-VersionEye-SBT

After fetching and parsing the build.sbt file successfully the file name turns into a link. You can enter the project detail view by clicking on the link. Here you can see which of your dependencies are out-dated and what is the current version.

02-VersionEye-SBT

By clicking on the Link “Visual Dependencies” VersionEye will show you all transitive dependencies, visualized as a super cool JavaScript Graph ūüôā

03-VersionEye-SBT

VersionEye will check your build.sbt file once a week and send you email notifications about out-dated dependencies. If everything is up-to-date you will get no email.

What is maven-compat ???

What is the maven-compat project? I was wondering. So I went to the official page of the project and found this: 

Screen Shot 2014-08-24 at 11.40.23

Maven2 classes maintained as compatibility layer.

¬†That is not very helpful! I would like to have a little bit more information! So clicked on the “Project Summary” link and got this.¬†

Screen Shot 2014-08-24 at 11.43.21

All right. Some auto generated site. Nothing I could not lookup in the pom.xml itself. And I still don’t know much more about this project. What does it do exactly? Why I need it?¬†

Unfortunately many Java projects are documented as bad as this one. Making an SMC public and auto generating a couple HTML pages doesn’t make it to a good Open Source project. Everytime I see something like this I get angry. Because the maintainers don’t give a shit about the community!

And by the way, JavaDocs are very poor docs! Everybody can generate JavaDocs. Why not put the project on GitHub or Bitbucket and provide a nice README with a red line, with a good intro into the project? That makes all the difference between a good and a bad open source project. 

Geek2Geek – Centralized Logging

Last week it happened again. Geek2Geek!

Geek2Geek_1_twitter

This time we came together at Flyeralarm in Berlin to talk about centralized logging. That is an interesting topic for all companies which have to scale. As soon you have more than 1 server you need to think about how you collect and analyze your log files in a distributed system. There are a couple good solutions out there for this problem.

Jilles van Gurp did the first talk about the ELKstack. ELK stands for E = Elasticsearch, L = Logstash and K = Kibana. All three products belong to the Elasticsearch company and they work all together smoothly in harmony. Jilles showed us how they use the ELK stack at Linko to build the LinkoApp.

Geek2Geek_2_linkoapp

Jilles gave us a short intro to the technology on a couple slides before he switched to the live demo. It was very interesting to listen to his real-world experiences with the ELK stack.

Geek2Geek_3_colectd

The learning from the past couple months are, it is easy to setup but you should be careful with the Elasticsearch cluster. Don’t shut it down all at once ūüėČ

After the first presentation the Pizza arrived and we took a little break with Pizza & Beer.

Geek2Geek_4_pizza

Lennart is THE guy behind Graylog2. He started the project a couples years ago at Jimdo. The very first version was implemented in Ruby. Graylog2 is a completely rewrite in Java. Lennart is also CoFounder of Torch, the company behind Graylog2.

Lennart gave a short intro about the history, intention and philosophy behind Graylog2.

Geek2Geek_5_Graylog2

I was impressed how much he knows about the other logging solutions, such as Logstash/Kibana and Splunk. He was not afraid to talk about feature comparisons and pros & cons of the different solutions.

Geek2Geek_6_Graylog2

Graylog2 is build for Enterprise usage. It is optimized for speed and high volume data. The interesting thing is that you can use it together with Elasticsearch and Logstash.

Many thanks for to Jilles and Lennart for the great talks. Both solutions are very interesting. If you still read logs on the server with “less” you should definitely check out these 2 great solutions!

By the way. I also tried to organize a Splunk talk, but unfortunately I couldn’t find any Splunkies¬†willing to give a talk about Splunk at Geek2Geek.

Many thanks to Flyeralarm for sponsoring Location, Pizza and Beer! You guys are awesome!

By the way Flyeralarm just opened a new branch in Berlin. They have a really nice office. This is their meeting room for example.

Screen Shot 2014-05-22 at 12.58.18

And they are currently looking for experienced PHP developers. If you are interested you should contact Thomas.

Comparison of Application Level Package Managers

I have to work with a lot (9) of different package managers at my daily work¬†at¬†VersionEye. Part of our mission is it¬†to make manual updating of dependencies extinct, because it‚Äôs a manual and time consuming task which nobody enjoys. That‚Äôs why we are building a notification system for open source software libraries to make Continuous Updating¬†easy and fun. And since we support several programming languages ‚Äď 8 at this point! ‚Äď I get to write crawlers and parsers for all of them. To give you a better overview over the strengths and weaknesses of these package managers, I picked the most popular one for each language and will compare them. The contenders are:

  • RubyGems /¬†Bundler (Ruby)
  • PIP / PyPI (Python)
  • Packagist / Composer (PHP)
  • NPM (Node.JS)
  • Bower (JS, CSS, HTML)
  • CocoaPods (Objective-C)
  • Maven (Java)
  • Lein (Clojure)

Comparison Matrix

Here are the results of my comparison.

Comparison of Package Managers

You can read the complete article on the VersionEye Blog and follow the discussion on hacker news and Reddit.

AbstractHttpClientWagon.getReadTimeout()I

I am currently writing a maven plugin and I got this error message by executing the goal.

[ERROR] Failed to execute goal group:artifact:1.0.0:goal (default-cli) on project artifact: Execution default-cli of goal group:artifact:1.0.0:goal failed: An API incompatibility was encountered while executing group:artifact:1.0.0:goal: java.lang.NoSuchMethodError: org.apache.maven.wagon.shared.http4.AbstractHttpClientWagon.getReadTimeout()I

I could resolve this after I updated from maven 3.0.3 to maven 3.0.5. The current version is solving this problem.

Getting a list of all Licenses in a Project

In a regular project you are using a handfull Software Libraries. Every Library in your project can use a different license. To get a list of all licenses which are used in your project can be difficult. You can double check the license for every single library in your project manually. But that is time intensive and a pain in the ass!

VersionEye is offering a quick solution for that. You can get the license list in less than 1 minute. After your login you are in “My Projects”. Click the link “Add New Project” in the left nav bar, to create a new project.

Screen Shot 2013-02-04 at 5.40.07 PM

Simply upload a project file. A pom.xml, Gemfile, composer.json, package.json, dependency.gradle or another supported file with your dependencies. After the upload you will see a tab with all the dependencies.

Screen Shot 2013-02-04 at 5.40.53 PM

Simply click on the “License” tab to see all licenses in this project. And here it is.

Screen Shot 2013-02-04 at 5.41.17 PM

In this example most Libraries are using the MIT License. Some of them are under “Ruby” license and for 1 Library VersionEye was not able to detect a license.

It’s that simple. Let me know if you have questions to this.

JsonIgnoreProperties

If you are using the Jackson project to parse JSON files than I recommend to use the “@JsonIgnoreProperties” Annotation. Assume that you have a JSON String like this, with 2 properties:

{
  "name" : "mike", 
  "sex" : "male"
}

If you want to parse that with Jackson, you have to create a class like this.


public class User {

private String name;
private String sex;

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }

  public String getSex() {
    return sex;
  }

  public void setSex(String sex) {
    this.sex = sex;
  }
}

And with Jackson you can map it like this:


ObjectMapper mapper = new ObjectMapper();
 User user = mapper.readValue(reader, User.class);

So far so good. As soon somebody adds a another Property to the JSON stream, your code breaks. Yes! ūüôā
Because Jackson tries to map every Property from the JSON to your class. Assume somebody ads the Property “age” and the JSON looks now like this:

{
  "name" : "mike", 
  "sex" : "male",
  "age" : "19"
}

Now Jackson tries to map “age” to the User class. And because their is no getter and setter your code will break with a nice Exception. Happy Exception Handling ūüôā
If you want to avoid this strange behavior, than put this Annotation on top of your User class: “@JsonIgnoreProperties(ignoreUnknown = true)”. With that Jackson will ignore unknown Properties.

I don’t know why I have to use this Annotation. For me it is pretty clear that an Exception occurs if I call an non existing setter. I mean ignoring unknown properties should be the default behavior. Everything else just doesn’t make any sense for me.

JsonAnySetter

I am using the Jackson JSON Lib in Java to process JSON. If you don’t know how JSON works in Ruby, you will like this project. Anyway. If you have dynamic JSON stream where the property names are not always the same, than you need the “@JsonAnySetter” Annotation. With that you can map dynamic¬†Properties¬†to a map.


private Map<String, Object> all = new HashMap<String, Object>();

@JsonAnySetter
public void set(String name, Object value) {
   all.put(name, value);
}

public Map<String, Object> getAll() {
   return all;
}

public void setAll(Map<String, Object> all) {
   this.all = all;
}

That worked out for me, for a String – String pair.

Automated Business Logic

If you are working with Hibernate this might be interesting for your. Automated Business Logic (ABL) is a Library which helps you to manage your Business Logic. It handles Cross Cutting Concerns and Multi Table Views. You just need to add a couple Annotations to your code. Take a look tot this demo here:

http://www.automatedbusinesslogic.com/live-demo

It just takes a couple minutes.

And here is ready to go Maven2 Archetype for ABL + Hibernate.

http://www.automatedbusinesslogic.com/maven-demo

You can follow ABL @VersionEye to get notifed about new releases.

http://versioneye.com/package/com~autobizlogic~abl–autobizlogic

Java Driver for Firebase

I just started a new Project @ GitHub, called jfirebase. That is basically a Java Wrapper for the REST API of Firebase. Clone the git repo and build the package with Maven2 / Maven3 like this:

mvn -Dmaven.test.skip=true package

There are Unit Tests for the project but If you want to run the test cases you have to add your firebase channel to the test classes.

You can install the JAR file with this command into your local Maven Repository:

mvn -Dmaven.test.skip=true install

The Driver contains basically 5 methods to interact with Firebase.

  • boolean write(Map<String, String> map)
  • Reader read(String uri)
  • boolean delete(String uri)
  • void setChannel(String channel)
  • String setKey(String key)

Here is an example for writing data to Firebase.

Map data = new HashMap();
data.put("firstname", "Robert");
data.put("lastname", "Reiz");
IDriver driver = new Driver();
driver.setChannel("http://YOUR_CHANNEL_AT_FIREBASE");
driver.write(data);

Your channel would be something like: “http://demo.firebase.com/myprojectname&#8221;.

OK. Here is an example for reading data.

IDriver driver = new Driver();
driver.setChannel("http://demo.firebase.com/SampleChat/");
Reader reader = driver.read("users/jack");
try{
   ObjectMapper mapper = new ObjectMapper();
   User user = mapper.readValue(reader, User.class);
   System.out.println(user.getName());
} catch (Exception ex){
   ex.printStackTrace();
}

The “read” method returns a “java.io.Reader”. With that you can do what you want. I am using here the jackson-core-lgpl JSON Mapper to map the JSON String from Firebase to my Java class User.

The delete method is pretty straight forward:

IDriver driver = new Driver();
driver.setChannel("http://demo.firebase.com/SampleChat/");
boolean deleted = driver.delete("user/jack");

Let me know if you have questions.

Maven2 Archetypes – Yes or No ?

A good friend of my is working on a cool project. A plugin for the Hibernate Persistence Framework. I am helping out with testing. And a couple days ago we had this discussion about how to get started with a project. I recommended to offer a Maven2 Archetype.

I asked Max if I can publish his E-Mail on my blog and take the conversation public. And here is his E-Mail:

———–

Hi Robert,
further thoughts on the Maven archetype idea. Having played with it, it doesn’t really seem practical because we don’t know what framework our users will be using. We could conceivably come up with an archetype for Grails, one for Spring, one for ZK, etc…
What we have done is Mavenize the basic demo so you can just download it and run it with mvn tomcat:run. That makes the zip file just 97k, which is nice. This is also true for the Grails demo.
I think it makes sense to make the tutorial a Maven project, but I don’t really see the value of making it an archetype. People will not want to create new projects based on this particular project — which uses plain old JSP. They’ll just get the basic project and fiddle with it while following the tutorial.
Your thoughts would be appreciated.
Thanks,
— Max

Uploading Artifacts to the central maven repository

Do you ever tried to upload an artifact to the central maven repository server? No? Well, that is not fun! The maven guys are pretty serious!

OK. If you want to upload something you “just” have to read this tutorial here:

http://maven.apache.org/guides/mini/guide-central-repository-upload.html

AND this super long tutorial here:

https://docs.sonatype.org/display/Repository/Sonatype+OSS+Maven+Repository+Usage+Guide

On maven.apache.org they tell you that the most easiest way to upload something is over sonatype. You just have to create a user in their JIRA bug tracking tool and open a ticket! *LOL* Are you serious? Publishing software by opening a ticket in a bug tracking tool? Really? Reaaaaalllllyyyyyyyy?

Are you fu**ing kidding me?

And the best part is, that your ticket will be handled in just 2 business days. Wow.

No wonder that every company and even small open source teams are setting up their own Maven2 Repository.

To publish a open source project in RubyGems.org took me less than 5 min and it was published¬†immediately. And I don’t know one single Ruby coder which is managing his own RubyGems Server.

Absent Code attribute

By executing the Unit Tests with TestNG for this Maven2 Archetype here:

https://robert-reiz.com/2012/02/18/archetype-richfaces-4-1-0-final-spring-3-1-0-release-hibernate-3-3-0/

I got this Exception:

Absent Code attribute in method that is not native or abstract in class file javax/faces/lifecycle/Lifecycle

This project is annotation driven. The JSF Managed Beans are annotated with Spring Annotations:

@Component
@Scope("request")

in the applicationContext.xml I have this lines here to read the annotations:

<context:annotation-config/>
<context:component-scan base-package="com.versioneye" />

I created a second applicationContext.xml for the test environment and modified the annotation-config. In the test environment I am scanning now just the peristence package. Because there are anyway no Unit Tests for the JSF Managed Beans.

<context:annotation-config/>
<context:component-scan base-package="com.versioneye.persistence"/>

Since I am not scanning the Managed Beans in the test environment I didn’t get the Exception above.

Archetype: RichFaces 4.1.0.Final + Spring 3.1.0.RELEASE + Hibernate 3.3.0

This maven2 archetype contains a little sample web application with this Frameworks:

  • Spring 3.1.0.RELEASE Framework
  • Servlet-API 2.5
  • JSF 2.0 (mojarra 2.1.2)
  • RichFaces 4.1.0.Final
  • ploinFaces 1.6
  • ploinMailFactory 1.3.1
  • Hibernate 3.3.0
  • TestNG 5.8
  • Log4J 1.2.15
  • HSQLDB 1.8.0.7

The configuration is annotation-driven. It is deployed on the PLOIN Repository-Server

http://www.ploin-m2.de/nexus/content/groups/public/

you can create a project from the archetype with the following command:

mvn archetype:generate -DarchetypeGroupId=org.ploin.archetype -DarchetypeArtifactId=demoSpringRichHibernate-archetype -DarchetypeVersion=1.2 -DarchetypeRepository=http://www.ploin-m2.de/nexus/content/groups/public/ -DgroupId=com.versioneye -DartifactId=myNewWebApp

The created project is a very simple web-application with a login mask and 2 xhtml-sites. I have tested the app on a tomcat 6.0.35 and Java 1.6 on Mac OS X Lion.

On the second page there is a small demonstration of the JSF 2.0 AJAX feature. Every time you type something in into the first input field, there is a whole server roundtrip. The ManagedBean gets updated immediately.

On the same page there is a small demo for the RichFaces calendar.

After the creation the app is running with the HypersoniceSQL DBMS. But it is very easy to switch to MySQL or Oracle. I put the drivers for MySQL and Oracle as a comment in the pom.xml. So you just need to comment in the right lines in the pom.xml.

EL Problems on Tomcat 6.0.35

Yesterday I wanted to start a new JSF archetype and I got this exception:

java.lang.LinkageError: loader constraint violation: when resolving interface method "javax.servlet.jsp.JspApplicationContext.getExpressionFactory()Ljavax/el/ExpressionFactory;" the class loader (instance of org/apache/catalina/loader/WebappClassLoader) of the current class, com/sun/faces/config/ConfigureListener, and the class loader (instance of org/apache/catalina/loader/StandardClassLoader) for resolved class, javax/servlet/jsp/JspApplicationContext, have different Class objects for the type javax/el/ExpressionFactory used in the signature

This happens if you have an “el-api.jar” in your project. The same jar file is already in the tomcat “lib” directory. I just removed the “el-api.jar” from my libs and than it worked out for me.

Unable to locate Spring NamespaceHandler for XML schema namespace

I am working on a backend Maven3 project. I am building an uber JAR which contains several other JAR files and resources. That worked pretty good. But than I tried to run the JAR file with

java -jar <uber-super-duper-jar-file>

I got this Exception:

INFO  org.springframework.beans.factory.xml.XmlBeanDefinitionReader  - Loading XML bean definitions from class path resource [applicationContext.xml]
Exception in thread "main" org.springframework.beans.factory.parsing.BeanDefinitionParsingException: Configuration problem: Unable to locate Spring NamespaceHandler for XML schema namespace [http://www.springframework.org/schema/context]

This is because I have several Spring Jar in my dependencies. Some of the spring jars contain meta info files with the same name. To avoid that some meta files are overridden you have to merge it. If you are using the maven shade plugin to build your JAR file you can do the merge with this xml snippet:

<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
  <resource>META-INF/spring.handlers</resource>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.AppendingTransformer">
  <resource>META-INF/spring.schemas</resource>
</transformer>

Here is a more detailed description:

http://maven.apache.org/plugins/maven-shade-plugin/examples/resource-transformers.html

Using Java in Rails with JRuby

JRuby is a pretty good Java implementation of Ruby. One of the biggest advantages of JRuby is that you can use Java Classes in your Ruby on Rails App. You can import JARs and take advantages of all the Java Libraries and Frameworks. In that way you don’t have to throw away your old Java Code. You can build a JAR and use it inside your new fancy Ruby App.

I recommend to build a Uber JAR, which contains all dependencies of your Java App. That is more easy to handle. You have to put your Uber JAR into the “lib” directory of your rails app. Than you have to import the JAR in the “application.rb”.

require "lib/your-java-all.jar"

That’s it! Know you can access the Classes inside the JAR. If you want to use a Java Class in a Controller you have to include the class:

include_class Java::org.trophic.graph.service.LocationServiceImpl
include_class Java::org.trophic.graph.factory.LocationFactory

And here is how you can use the Java Classes:

locationService = LocationFactory.locationService
locations = locationService.locations

And here is the full example of the controller:

class PageController < ApplicationController

  include_class Java::org.trophic.graph.service.LocationServiceImpl
  include_class Java::org.trophic.graph.factory.LocationFactory

  def home
      locationService = LocationFactory.locationService
      @locations = locationService.locations
  end

end

It is super easy! Isn’t it?

JRuby + Spring + AspectJ + Neo4J

I am playing around with JRuby and Neo4j as Graph Engine. My set up on the backend side contains a Java Application with Spring Annotations, AspectJ and Neo4J. In the frontend I have a Rails App with JRuby.

Any time I try to access a class that have an annotation like “@NodeEntity”, I get an Exception. It seems that Spring and AspectJ is to much in this chain. I should get rid of Spring and AspectJ.