Java HTTP Request with Basic Auth

This is how you do a simple HTTP request with Java. These code performs the actual HTTP request and saves the response in a String variable.

URL url = new URL(address);
URLConnection conn = url.openConnection();
conn.setConnectTimeout(30000); // 30 seconds time out
String line = "";
StringBuffer sb = new StringBuffer();
BufferedReader input = new BufferedReader(new InputStreamReader(conn.getInputStream()) );
while((line = input.readLine())!=null)
  sb.append(line);
input.close();
String response = sb.toString();

If the HTTP server requires Baisc Auth this code will fail. To make it work for Basic Auth these 3 additional lines are required.

String user_pass = username + ":" + password;
String encoded = Base64.encodeBase64String( user_pass.getBytes() );
conn.setRequestProperty("Authorization", "Basic " + encoded);

The whole method looks like that:

public String getHttpResponse(String address, String username, String password) throws Exception {
  URL url = new URL(address);
  URLConnection conn = url.openConnection();
  conn.setConnectTimeout(30000); // 30 seconds time out

  if (username != null && password != null){
    String user_pass = username + ":" + password;
    String encoded = Base64.encodeBase64String( user_pass.getBytes() );
    conn.setRequestProperty("Authorization", "Basic " + encoded);
  }

  String line = "";
  StringBuffer sb = new StringBuffer();
  BufferedReader input = new BufferedReader(new InputStreamReader(conn.getInputStream()) );
  while((line = input.readLine()) != null)
    sb.append(line);
  input.close();
  return sb.toString();
}

How to get notified about out-dated dependencies in build.sbt?

Scala SBT is the build tool for the programming language Scala. You can specify your project dependencies in the “build.sbt” file, similar to the pom.xml file in Maven. A build.sbt file can look like this.

name := "s3crate"

organization := "codexica"

version := "0.0.1-SNAPSHOT"

scalaVersion := "2.10.2"

libraryDependencies += "net.java.dev.jets3t" % "jets3t" % "0.9.0"

libraryDependencies += "com.amazonaws" % "aws-java-sdk" % "1.3.33"

But how do you get notified about new versions of your dependencies?

VersionEye now supports Scala SBT. VersionEye can monitor your build.sbt file on GitHub / Bitbucket. If you are logged in to VersionEye you just have to navigate to your GitHub / Bitbucket repository and flip the switch beside the build.sbt file.

01-VersionEye-SBT

After fetching and parsing the build.sbt file successfully the file name turns into a link. You can enter the project detail view by clicking on the link. Here you can see which of your dependencies are out-dated and what is the current version.

02-VersionEye-SBT

By clicking on the Link “Visual Dependencies” VersionEye will show you all transitive dependencies, visualized as a super cool JavaScript Graph 🙂

03-VersionEye-SBT

VersionEye will check your build.sbt file once a week and send you email notifications about out-dated dependencies. If everything is up-to-date you will get no email.

What is maven-compat ???

What is the maven-compat project? I was wondering. So I went to the official page of the project and found this: 

Screen Shot 2014-08-24 at 11.40.23

Maven2 classes maintained as compatibility layer.

 That is not very helpful! I would like to have a little bit more information! So clicked on the “Project Summary” link and got this. 

Screen Shot 2014-08-24 at 11.43.21

All right. Some auto generated site. Nothing I could not lookup in the pom.xml itself. And I still don’t know much more about this project. What does it do exactly? Why I need it? 

Unfortunately many Java projects are documented as bad as this one. Making an SMC public and auto generating a couple HTML pages doesn’t make it to a good Open Source project. Everytime I see something like this I get angry. Because the maintainers don’t give a shit about the community!

And by the way, JavaDocs are very poor docs! Everybody can generate JavaDocs. Why not put the project on GitHub or Bitbucket and provide a nice README with a red line, with a good intro into the project? That makes all the difference between a good and a bad open source project. 

Geek2Geek – Centralized Logging

Last week it happened again. Geek2Geek!

Geek2Geek_1_twitter

This time we came together at Flyeralarm in Berlin to talk about centralized logging. That is an interesting topic for all companies which have to scale. As soon you have more than 1 server you need to think about how you collect and analyze your log files in a distributed system. There are a couple good solutions out there for this problem.

Jilles van Gurp did the first talk about the ELKstack. ELK stands for E = Elasticsearch, L = Logstash and K = Kibana. All three products belong to the Elasticsearch company and they work all together smoothly in harmony. Jilles showed us how they use the ELK stack at Linko to build the LinkoApp.

Geek2Geek_2_linkoapp

Jilles gave us a short intro to the technology on a couple slides before he switched to the live demo. It was very interesting to listen to his real-world experiences with the ELK stack.

Geek2Geek_3_colectd

The learning from the past couple months are, it is easy to setup but you should be careful with the Elasticsearch cluster. Don’t shut it down all at once 😉

After the first presentation the Pizza arrived and we took a little break with Pizza & Beer.

Geek2Geek_4_pizza

Lennart is THE guy behind Graylog2. He started the project a couples years ago at Jimdo. The very first version was implemented in Ruby. Graylog2 is a completely rewrite in Java. Lennart is also CoFounder of Torch, the company behind Graylog2.

Lennart gave a short intro about the history, intention and philosophy behind Graylog2.

Geek2Geek_5_Graylog2

I was impressed how much he knows about the other logging solutions, such as Logstash/Kibana and Splunk. He was not afraid to talk about feature comparisons and pros & cons of the different solutions.

Geek2Geek_6_Graylog2

Graylog2 is build for Enterprise usage. It is optimized for speed and high volume data. The interesting thing is that you can use it together with Elasticsearch and Logstash.

Many thanks for to Jilles and Lennart for the great talks. Both solutions are very interesting. If you still read logs on the server with “less” you should definitely check out these 2 great solutions!

By the way. I also tried to organize a Splunk talk, but unfortunately I couldn’t find any Splunkies willing to give a talk about Splunk at Geek2Geek.

Many thanks to Flyeralarm for sponsoring Location, Pizza and Beer! You guys are awesome!

By the way Flyeralarm just opened a new branch in Berlin. They have a really nice office. This is their meeting room for example.

Screen Shot 2014-05-22 at 12.58.18

And they are currently looking for experienced PHP developers. If you are interested you should contact Thomas.

Comparison of Application Level Package Managers

I have to work with a lot (9) of different package managers at my daily work at VersionEye. Part of our mission is it to make manual updating of dependencies extinct, because it’s a manual and time consuming task which nobody enjoys. That’s why we are building a notification system for open source software libraries to make Continuous Updating easy and fun. And since we support several programming languages – 8 at this point! – I get to write crawlers and parsers for all of them. To give you a better overview over the strengths and weaknesses of these package managers, I picked the most popular one for each language and will compare them. The contenders are:

  • RubyGems / Bundler (Ruby)
  • PIP / PyPI (Python)
  • Packagist / Composer (PHP)
  • NPM (Node.JS)
  • Bower (JS, CSS, HTML)
  • CocoaPods (Objective-C)
  • Maven (Java)
  • Lein (Clojure)

Comparison Matrix

Here are the results of my comparison.

Comparison of Package Managers

You can read the complete article on the VersionEye Blog and follow the discussion on hacker news and Reddit.

AbstractHttpClientWagon.getReadTimeout()I

I am currently writing a maven plugin and I got this error message by executing the goal.

[ERROR] Failed to execute goal group:artifact:1.0.0:goal (default-cli) on project artifact: Execution default-cli of goal group:artifact:1.0.0:goal failed: An API incompatibility was encountered while executing group:artifact:1.0.0:goal: java.lang.NoSuchMethodError: org.apache.maven.wagon.shared.http4.AbstractHttpClientWagon.getReadTimeout()I

I could resolve this after I updated from maven 3.0.3 to maven 3.0.5. The current version is solving this problem.

Getting a list of all Licenses in a Project

In a regular project you are using a handfull Software Libraries. Every Library in your project can use a different license. To get a list of all licenses which are used in your project can be difficult. You can double check the license for every single library in your project manually. But that is time intensive and a pain in the ass!

VersionEye is offering a quick solution for that. You can get the license list in less than 1 minute. After your login you are in “My Projects”. Click the link “Add New Project” in the left nav bar, to create a new project.

Screen Shot 2013-02-04 at 5.40.07 PM

Simply upload a project file. A pom.xml, Gemfile, composer.json, package.json, dependency.gradle or another supported file with your dependencies. After the upload you will see a tab with all the dependencies.

Screen Shot 2013-02-04 at 5.40.53 PM

Simply click on the “License” tab to see all licenses in this project. And here it is.

Screen Shot 2013-02-04 at 5.41.17 PM

In this example most Libraries are using the MIT License. Some of them are under “Ruby” license and for 1 Library VersionEye was not able to detect a license.

It’s that simple. Let me know if you have questions to this.