poniedziałek, 15 czerwca 2015

Java 8 lambdas and checked exceptions

The Stream API in Java 8 is a blast. Finally, it is possible to easily write concise, declarative code just with pure JDK. There is one problem though - the limitation that lambdas cannot throw checked exceptions. I am sure there are plenty of good reasons not to do such terrible thing, but honestly: it just pisses off. So, I should do something with it then. First attempt is to wrap the lambda's body with a function which would turn the exception from checked to unchecked one (assuming the TechnicalException extends RuntimeException):

public static <T, R> Function<T, R> unchecked(FunctionWithException<T, R> f) {
  Function<T, R> rf = p -> {
    try {
      return f.invokeIt(p);
    } catch (Exception e) {
      throw new TechnicalException("Exception within lambda expression:", e);
    }
  };
  return rf;
}

and write something like this:

List<File> filesList = ...
filesList.stream()
  .map( unchecked( file -> FileUtils.readFileToString(file, "UTF-8" ) ) )
  .map( .... );

but, it obviously is not working, since the Function::apply(T t) does not throw an exception according to the signature. The solution is to define any other @FunctionalInterface just matching the number of arguments and use it instead, like this:

public static <T, R> Function<T, R> unchecked(FunctionWithException<T, R> f) {
  Function<T, R> rf = p -> {
    try {
      return f.invokeIt(p);
    } catch (Exception e) {
      throw new TechnicalException("Exception within lambda expression:", e);
    }
  };
  return rf;
}

Unfortunatelly, depending on the later usage, we may need multiple versions for different interfaces from java.util.function, but we can hide this ughliness in some class :)

czwartek, 29 sierpnia 2013

Legacy Java web apps and Unicode support

I had a problem with a web app that was not handling Unicode strings (Japanese, Korean) correctly.
The thing was, that the Tomcat (7.0.4x) container I was deploying the app into, was (incorrectly) using the ISO-8859-1 encoding for request parameters.

According to the spec, it could be overwritten in conf/server.xml for each Connector with the
 URIEncoding="UTF-8" attribute.

Tried it - had no effect ;( I have also found out some other comments in net that it didn't worked for other people as well.

So, I went for manual setting the encoding with:

  ServletRequest.setCharacterEncoding("UTF-8");

the problem was, though, that this has to be invoked BEFORE any request.getParameter() method is invoked.
Unfortunately, I had to build my app on top of some crappy webapp a customer had and I couldn't modify the core code. And, of course, this 'core' was playing with request.getParameter() very early in servlet's lifecycle.

So, the solution I come up was just to create a filter which sets the encoding, which is set up early (before my app) in the servlet chain:

web.xml:

...
    <filter>
        <filter-name>fixEncodingFilter</filter-name>
        <filter-class>pack.age.FixEncodingFilter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>fixEncodingFilter</filter-name>
        <url-pattern>/io/*</url-pattern>
    </filter-mapping>
...


This fixed the problem, nice and clean.

środa, 12 czerwca 2013

Limiting outgoing connections by max TPS

Interesting problem showed up. A customer requires that we never exceed some agreed number of TPS against his service (SOAP WS). On our side we've got couple of instances generating the traffic. They  in fact act as a pipe, processing the incoming requests. It seems a bit tricky to get all the instances to communicate with themselves extremely quickly to exchange the TPS information each of them has (sure, can be done - and I would use ØMQ for that if needed).
To sum up -  we have no influence on the incoming load, which comes from another customer (it's Google, actually ;)). Still, we have to stop all our requests to be forwarded. We need to start rejecting them above the threshold - queuing waiting for the less load would leave us with growing latency and with exhausted http connection pool in front of our app.
So, the idea is to use (already existing) reverse proxy  between the app and the outgoing customer's service. The xinetd's config comes in handy (from man page):

    cps

Limits the rate of incoming connections. Takes two arguments. The first argument is the number of connections per second to handle. If the rate of incoming connections is higher than this, the service will be temporarily disabled. The second argument is the number of seconds to wait before re-enabling the service after it has been disabled. The default for this setting is 50 incoming connections and the interval is 10 seconds.

The plan is to set it up to 'cps 29 0' (the goal is never exceed 30TPS) and see what happens :)

Different setting for the interval would result, in case of constant traffic like 31TPS, with loosing all 31 requests every 2nd second. Not good.

OK, let's see if the interval '0' works as expected... I'll update with results.


UPDATE:
finally, I have found  xinetd useless for the task. The thing is that (just took a look into the source code), when cps condition is reached, the service is stopped. And by this I mean that it is literally totally stopped... it rejects everything.  Actually, with that knowledge and reading the above spec now I see it states exactly that :) But still - it was surprising - I was rather expecting it to stop accepting more connections,  not to drop all undergoing work.

Well... the final solution is the most simple one: the TPS is monitored by the app itself. It is running on 2 separate instances, not aware about each other.  Both are set to work with max 14TPS. Of course it won't ever meet the max 30TPS, because it now all depends on the load balancer before the apps... Anyway, it is a conscious decision of the customer which claims that it will never reach so high rate anyway, so why should I care more than them? "Customer is King" :) - I did my job clearly explained all the consequences.

I have just used the com.google.common.util.concurrent.RateLimiter (from Google's Guava lib):

public synchronized static void checkIfBelowMaxTPS(final FRBYGConfig config, final String provider) throws MaxLimitProcessingException {

        RateLimiter rateLimiter = provider2RateLimiterMap.get(provider);

        //--- just for a double-check/logging;
        long now = System.currentTimeMillis(); //watch-out - different precision depending on OS (e.g. 10ms steps)
        requestsSet.add(now);
        Long oneSecBefore = now-1000-1; //-1 is because the subSet() is exclusive
        SortedSet<Long> subset = requestsSet.subSet(oneSecBefore, Long.MAX_VALUE);
        int noReqWithin1Sec = subset.size();
        requestsSet = subset;
        //---

        log.info("checking if the TPS is not above the treshold (provider="+provider+", limit="+rateLimiter.getRate()+"tps" + " - the current requests rate is: "+noReqWithin1Sec+"tps");
        if (!rateLimiter.tryAcquire()) {
            throw new MaxLimitProcessingException(...);
        }
    }


Also, just for a double check and logging purpose (RateLimiter doesn't provide runtime stats usable for logging) I added separate calculations there. It usually agree with RateLimiter, but is not 100% accurate. I have decided to trust RateLimiter in this task though.


piątek, 7 czerwca 2013

A new (ergonomic) keyboard

I decided to get myself a treat and to ease my aches caused by long hours of typing and get a new super-comfortable keyboard.

I looked up a possibilities first - it really seems there is just a couple of such things in the marked now, I was really surprised. One can find lots of gamer-dedicated mice, keyboards, special additional keyboards etc. But, there are just a few ergonomic keyboards meant for work. And - they are awfully expensive. My dream keyboard would be one of the Kinesis products e.g. this one:


But, the price - $300? WTF? Just for the simpler version...

So, surprisingly, I ended up with a Microsoft's keyboard - Natural Ergonomic 4000.




I have to say it is pretty nice, the price is OK as well (under 50Eur). I took the cable version over the wireless - batteries pisses me off and I work usually by the desk. The lining under wrists is a soft cushion, the special keys are working flawlessly under Linux - the Favorites, Home, Search, Back, Forward, Play, Volume controls. Really comfy, the angle the 2 pieces are set could be regulated though. I don't think I can manage to make the 'Zoom' switch to work, since xav doesn't show any keyboard input on them. Pity.
There are some flaws though - one is that the spacebar work not smoothly when pressed on the left/right edge (and it is meant to be used like this). The printings on the keys is not precise (like the paint got smudged). Overall - I like it, the paint I don't care since I do not look at the keyboard while typing, the spacebar seems to get working better with time. Now I have to try to use this pod in front edge of the keyboard - seems like a bit of revolution to the overall posture, but it would require sitting higher a bit (or have lower desk). I'll try.

The great thing I love is that the 'Microsoft' key is big and it is my mod key in Xmonad :)

Ecllipse 4.2 - slow as hell

I have just installed the brand new Eclipse 4.2 Juno. It is pretty (who cares!?) but - it is soooo slower than the previous 3.x version I had (Indigo).
Looked up what I can do - unfortunately this seems to be a result of a new UI feature - a user can now use CSS to modify the look&feel. It came with a price though.

I started up with modyfying the eclipse.ini the following way:

-startup
plugins/org.eclipse.equinox.launcher_1.3.0.v20120522-1813.jar
--launcher.library
plugins/org.eclipse.equinox.launcher.gtk.linux.x86_64_1.1.200.v20120913-144807
-showsplash
org.eclipse.platform
--launcher.XXMaxPermSize
256m
--launcher.defaultAction
openFile
-vm
/opt1/soft/java/jdk1.7.0_13/bin/java
-vmargs
-Xmn128m
-Xms1024m
-Xmx1024m
-Xss2m
-XX:PermSize=128m
-XX:MaxPermSize=128m
-XX:+UseParallelGC


It significantly improved the start-up time and an overall responsiveness.
Still - it is terrible.

Then, I switched to Classic look&feel (Preferences->General->Appearance) and made sure animations are disabled. It has improved things - just a tad.

Anyway, GUI is still awfully slow (switching tabs, editing pom.xml) comparing to the previous version. 

Looked up in the net - seems there's nothing which can be done.
Just to make things clear - my machine is pretty fast:  i7, 8 cores, 8GB RAM, SSD.


The other surprise  - the Content Assists didn't work for Java code!
Seems it was just disabled by default - Preferences>Java>Editor>Content Assist>Advanced... WTF? 

Just downloaded Intellij Idea... 

piątek, 31 maja 2013

Import a private key in PEM format to Java keystore

I got a key and a certificate in PEM format and I need to put it into a Java keystore (for WS security - message signing). Java's keytool doesn't let you to import a private key, normally one just creates a keystore with a new private key in one shot. I need to do this with a key and cert I have got from non-Java folks...
Since I just had to do it again and I managed to forget how I had this done at the first time - I am adding this post here...

First step is to convert both of them to DER format with following commands:

openssl pkcs8 -topk8 -nocrypt -in private_key.key -inform PEM -out  private_key.der -outform DER
openssl x509 -in server_cert.crt -inform PEM -out server_cert.der -outform DER

Now, to make a keystore out of it:

java ImportKey private_key.der server_cert.der alias_priv_key

(This is a tool I found somewhere in the net, wich may be downloaded from ImportKey.java. No dependencies outside JDK.)

the output is:

Using keystore-file : /home/bartek/keystore.ImportKey
One certificate, no chain.
Key and certificate stored.
Alias: alias_priv_key  Password:importkey

next step is to change the keystore's password with:

keytool -storepasswd -keystore keystore.ImportKey

give the 'importkey' password, then enter the new one twice and the job is done.

If the task is to add the key to already existing keystore with some other certs/keys - you need to merge them together (but luckily I don't need that :) ). 

czwartek, 30 maja 2013

Web server slow start problem (Jetty/Tomcat)


Today I started to feel I am having a problem with my maven and Jetty configuration. It started slower and slower, without any clear reason. Simple web app consisting of just a 5 SOAP endpoints (CXF) started in about 1 minute.
So, I started to investigate it and found the reason - seems that the container was looking for Servlet 3.0 annotations in the whole classpath during the start. This was completely unnecessary in my case, since I am not using any of them.
To stop that - it is possible to inform the container in web.xml declaration, that our configuration is complete and there is no need to scan for it.

 <web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         metadata-complete="true"
         version="3.0">

After that, startup time went down to couple of seconds - voilà!

The nice thing is that the metadata-complete="true" indicates that the JAR files in /WEB-INF/lib doesn't need to be scanned for annotations, but the webapp's own classes will still be scanned.