Log4j 2 asserting log entries with Junit

Lets start with the usage in JUnit

@Rule public LogAccessor logAccessor=new LogAccessor();

@Test @LogAccessorLogLevel(level = "DEBUG", category = "com.cedricwalter")
public void act_arrange_assert() {
    // Arrange          
    // Act           
    // Assert      
    logAccessor.assertCount(1).assertLevel(Level.DEBUG).assertMessage("Hellow World");
}

Notes
LogAccessorLogLevel is optionnal

Define a new annotation LogAccessorLogLevel .java:

package com.cedricwalter.logging;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(value = RetentionPolicy.RUNTIME)
@Target(value = {ElementType.METHOD})
public @interface LogAccessorLogLevel {
    String level() default "ERROR";

    String category() default "";
}

Create a new appender TestAppender.java

package com.cedricwalter.logging;

import org.apache.logging.log4j.core.Filter;
import org.apache.logging.log4j.core.Layout;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.core.appender.AbstractAppender;

import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;

public class TestAppender extends AbstractAppender {
    private final List log = new ArrayList<>();

    protected TestAppender(String name, Filter filter, Layout<? extends Serializable> layout) {
        super(name, filter, layout);
    }

    public List getLog() {
        return new ArrayList<>(log);
    }

    @Override
    public void append(LogEvent logEvent) {
        log.add(logEvent);
    }
}

add the Rule class

package com.cedricwalter.logging;

import org.apache.logging.log4j.Level;
import org.junit.rules.TestWatcher;
import org.junit.runner.Description;

import java.util.List;

import static com.innoveo.skye.common.utils.matcher.RegexMatcher.matches;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.is;

public class LogAccessor extends TestWatcher {
    private TestAppender appender;
    private LogAccessorLogLevel logAccessorLogLevel;

    private List getLog() {
        return getAppender().getLog();
    }

    public LogAccessor assertCount(int expectedLogEntryCount) {
        assertThat(getLog().size(), is(expectedLogEntryCount));
        return this;
    }

    public LogAccessor assertLevel(Level expectErrorLevel) {
        assertThat(getLog().get(0).getLevel(), is(expectErrorLevel));
        return this;
    }

    public LogAccessor assertMessage(String expectedMessage) {
        assertThat(getLog().get(0).getMessage().getFormattedMessage(), matches(expectedMessage));
        return this;
    }

    @Override
    protected void starting(Description description) {
        appender = new TestAppender("testAppender", null, null);
        Level level = getLevel(description);          /
        /Add appender to root logger
        org.apache.logging.log4j.core.Logger rootLogger = (org.apache.logging.log4j.core.Logger) LogManager.getRootLogger();
        Configuration configuration = rootLogger.getContext().getConfiguration();
        configuration.addLoggerAppender(rootLogger, appender);
        String logCategory = getLogCategory(description);
        if (logCategory != null) {
            changeLoggerLevel(logCategory, level, appender);
        }
    }

    /**
     * The problem was with the getLoggerConfig() call;
     * if the module you are trying to give a new level is not yet
     * * registered, this method returns the root logger (or any intermediate sub path registered), and thus instead
     * * of altering the level for com.mycompany you will alter root or com level. That's why you have to add a new
     * * LoggerConfig in case the module to alter is not yet registered.
     * *
     * * @param module
     * * @param level      * @param appender
     */
    private static void changeLoggerLevel(final String module, final Level level, TestAppender appender) {
        LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
        AbstractConfiguration configuration = (AbstractConfiguration) ctx.getConfiguration();
        LoggerConfig loggerConfig = configuration.getLogger(module);
        if (loggerConfig != null) {
            org.apache.logging.log4j.core.Logger logger = (org.apache.logging.log4j.core.Logger) LogManager.getLogger(module);
            configuration.addLoggerAppender(logger, appender);
            loggerConfig.setLevel(level);
        } else {
            loggerConfig = new LoggerConfig(module, level, true);
            configuration.addLogger(module, loggerConfig);
            ctx.updateLoggers(configuration);
            org.apache.logging.log4j.core.Logger logger = (org.apache.logging.log4j.core.Logger) LogManager.getLogger(module);
            configuration.addLoggerAppender(logger, appender);
            loggerConfig.setLevel(level);
        }
        ctx.updateLoggers(configuration);
    }

    @Override
    protected void finished(Description description) {
        removeAppender(LogManager.ROOT_LOGGER_NAME);
        String logCategory = getLogCategory(description);
        if (logCategory != null) {
            removeAppender(logCategory);
        }
    }

    private void removeAppender(String loggerName) {
        org.apache.logging.log4j.core.Logger logger = (org.apache.logging.log4j.core.Logger) LogManager.getLogger(loggerName);
        Configuration configuration = logger.getContext().getConfiguration();
        LoggerConfig loggerConfig = configuration.getLoggerConfig(loggerName);
        loggerConfig.removeAppender(appender.getName());
    }

    private TestAppender getAppender() {
        return appender;
    }

    private Level getLevel(Description description) {
        logAccessorLogLevel = description.getAnnotation(LogAccessorLogLevel.class);
        if (logAccessorLogLevel != null) {
            return Level.toLevel(logAccessorLogLevel.level());
        }
        return Level.ERROR;
    }

    private String getLogCategory(Description description) {
        logAccessorLogLevel = description.getAnnotation(LogAccessorLogLevel.class);
        if (logAccessorLogLevel != null) {
            return logAccessorLogLevel.category().getLoggerName();
        }
        return null;
    }

    @Override
    public String toString() {
        LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
        StringBuilder str = new StringBuilder();
        for (LoggerConfig logger : ctx.getConfiguration().getLoggers().values()) {
            final String loggerName = LogManager.ROOT_LOGGER_NAME.equals(logger.getName()) ? "Root Logger" : logger.getName();
            str.append("Found logger '" + loggerName + "' with level " + logger.getLevel());
        }
        return str.toString();
    }
}

Implement SSH port knocking IN linux

Port Knocking is a technique to secure services behind a firewall until a specific knock sequence is given. Once that sequence is given, the IP address that initiated the knock may be allowed to access the service for a short period of time. A knocking server listens to all traffic on an Ethernet (or PPP) interface, looking for special “knock” sequences of port-hits. A client makes these port-hits by sending a TCP (or UDP) packet to a port on the server.

This is a bit paranoid, but it add another layer of security, an attacker will have either to try all ports combinations or know the secret combination (knock) to be able to connect to your SSH daemon for example.

First, you’ll have to be sure to have a port knocking client, or you will be kick out forever of your own server!

Continue reading Implement SSH port knocking IN linux

Initial Coin Offering in most relevant countries

 This new rendering will allow you to better compare countries

The rendering being dynamic you can also pass some parameters like

https://ico.tokens-economy.com/statistics/collage.html?width=800&height=800

The default size is width=450&height=450

Filter by year ico end. e.g all ico ended in 2017

https://ico.tokens-economy.com/statistics/collage.html?year=2017 

Filter by category and year

https://ico.tokens-economy.com/statistics/collage.html?category=Cryptocurrency 

And more

  • ICO with no raised amount is also displayed now.
  • ICO webpage has been added
  • Text in bubble scale now properly and are always centered

Running WordPress and MySql in Docker

Notes

  • A Docker image is a lightweight, stand-alone, executable package of a piece of software that includes everything needed to run it: code, runtime, system tools, system libraries, settings. Available for both #Linux and #Windows based apps, containerized software will always run the same, regardless of the environment. #Containers isolate software from its surroundings, for example differences between development and staging environments and help reduce conflicts between teams running different software on the same infrastructure.
  • A container is a runtime instance of an image—what the image becomes in memory when actually executed. It runs completely isolated from the host environment by default, only accessing host files and ports if configured to do so. #Container are stateless and should be considered read only! (you can go inside the container change data, but at container creation your changes are lost)
  • Both MYSQL (Data, users) and WordPress (plugins, themes, uploads) are stateful, so we have to use #Docker volume to persist data across container restart.
  • expose: 3306 will let you connect later with MySQLWorbench to the port from outside of the container. it is optional.
  • depends_on tell #wordpress to wait till mysql db container is up
  • The always restart policy tells #Docker to restart the container under every circumstance. What’s great about the always restart policy is that even if our #Docker host was to crash on boot, the Docker service will restart our container.

We will be using the official #Wordpress image from Docker HUB But first let’s create a file uploads.ini to avoid issues later on while uploading plugins in wordpress.

file_uploads = On 
memory_limit = 256M 
upload_max_filesize = 256M 
post_max_size = 300M 
max_execution_time = 600 

Create a new file docker-compose.yml, adapt values to your liking, especially all passwords and username. Note that #MYSQL and #Worpress data are persisted OUTSIDE of container.

version: '2' 
services: 
 wordpress: 
	depends_on: 
	 - db 
	image: wordpress:latest 
	volumes: 
	 - ./uploads.ini:/usr/local/etc/php/conf.d/uploads.ini 
	 - ./file-wordpress:/var/www/html 
	ports: 
	 - 80:80 
	 - 443:443 
	restart: always 
	environment: 
	  WORDPRESS_DB_HOST: db 
	  WORDPRESS_DB_USER: wordpress 
	  WORDPRESS_DB_PASSWORD: wordpress 
	  WORDPRESS_TABLE_PREFIX: abcd 
 db: 
	image: mysql:5.7 
	volumes: 
	 - ./db-wordpress:/var/lib/mysql 
	restart: always 
	expose: 
	 - "3306" 
	environment: 
	  MYSQL_ROOT_PASSWORD: wordpress 
	  MYSQL_DATABASE: wordpress 
	  MYSQL_USER: wordpress 
	  MYSQL_PASSWORD: wordpress

The docker-compose up command aggregates the output of each container. It Builds, (re)creates, starts, and attaches to containers for a service. When the command exits, all containers are stopped. Running docker-compose up -d starts the containers in the background and leaves them running. To start WordPress in the background

docker-compose up -d

To find the name of the container

docker ps -a

or

docker-compose ps

To read the logs file

docker logs wordpress docker logs db

To go inside the container (remember all changes in there are lost at container restart)

docker exec -it wordpress bash

To delete all volume

docker-compose  rm -v

git-branch-renamer-maven-plugin

When working with many feature/release/bugix/hotfix branches, it is a bad idea to start changing the pom version as this will create merge conflicts using pull request. this plugin allow you to keep in ALL branches the same pom version for all your projects, for example MASTER-SNAPSHOT the version will be derived from branch name automagically 🙂

You may want to read more first these 2 short articles

git-branch-renamer-maven-plugin allow you to keep in ALL branches the same pom version for all your projects: for example MASTER-SNAPSHOT and never change it again.

the project version will be derived from branch name automatically when running in your continuous integration server.

branch name feature/xxxx

  • <version>xxxx-SNAPSHOT</version> (default)
  • <version>xxxx</version> (release = true)
  • <version>0-xxxx-SNAPSHOT</version> (forceNumericalVersion = true)
  • <version>feature-xxxx-SNAPSHOT</version> (filterOutBranchQualifier = false)

The project is hosted at Github https://github.com/cedricwalter/git-branch-renamer-maven-plugin 

Java Networking and Proxies with HTTPS

In today’s networking environments, particularly corporate ones, application developers have to deal with proxies almost as often as system administrators. In some cases the application should use the system default settings, in other cases it will we want to have a very tight control over what goes through which proxy, and, somewhere in the middle, most applications will be happy to delegate the decision to their users by providing them with a GUI to set the proxy settings, as is the case in most browsers.

There is a lot of misleading information on the internet on how to not use the proxy when using HTTPS  connections. There are 3 properties you can set to specify the proxy that will be used by the http protocol handler:

  • http.proxyHost / https.proxyHost : the host name of the proxy server
  • http.proxyPort / https.proxyPort: the port number, the default value being 80.
  • http.nonProxyHosts:a list of hosts that should be reached directly, bypassing the proxy. This is a list of patterns separated by ‘|’. The patterns may start or end with a ‘*’ for wildcards. Any host matching one of these patterns will be reached through a direct connection instead of through a proxy.

There is no https.nonProxyHosts, if your company only use https backend connections (which is highly recommended even for internal network connections) then you may have to exclude your service xxx.xxx.xxx.xxx with something like this

-Dhttp.nonProxyHosts="localhost|127.0.0.*|xxx.xxx.xxx.xxx" -Dhttps.proxyHost=proxy.xxxx.xxx -Dhttp.proxyPort=443

For the “non proxy hosts” list, the HTTPS protocol handler will use the same as the http handler (i.e. http.nonProxyHosts).

References http://docs.oracle.com/javase/8/docs/technotes/guides/net/proxies.html 

maven-release-plugin with GIT

Problem with the #maven release plugin

You may recognize yourself with the following use-cases:

  • We have several big modules/repositories with many branches. Per repository we have one parent pom and 50+ children poms. This is for a large JEE project (400 K lines of code) that it is delivered to several different clients.
  • We create feature branches, bug fixes, integration branches, release branches etc. We have many versions of the same application deployed to different production environments.
  • Every time a new branch is created, the version tag in the parent pom and all children poms need to be updated
  • When these branches need to be merged all these poms cause conflicts and they need to be manually resolved, because some times there are other changes in the poms besides just the version change. It is a major problem for us. 
  • We change the version on every branch because we have a continuous integration server that builds every branch and runs regression tests against every branch, therefore the versions must be different per branch.
  • Being able to keep the new version isolated to one file (the parent pom) really makes sense here. (Child pom inherits version from parent property)
  • We need to use SNAPSHOTS as we cannot have every developer build the whole system. Developers need to use artifacts built by #TeamCity for their feature branch and they cannot be changing to the last build version they depend on every time a new development build is created.

The #maven release plugin create a lot of issues that have been around for many many years.

Continue reading maven-release-plugin with GIT

Review: Getting Started with Apache Maven by Russell Gold

Some time ago I was asked if I would like to write a review about one of the new video courses from Packt Publishing. It was “Getting Started with Apache #Maven” http://bit.ly/1fycmpP by Russell Gold and since I have been using Maven for some years now (since 2007) and did publish some articles myself, I thought it would be nice to help them promote Apache #Maven.

The course is organized in eight chapters, forty videos with a length between two and five minutes, for a total length of two hours. The aim of the course is to provide the shortest path to use effectively Maven

You can download the course as a single zip file. After unpacking the file to a local hard disk you find a user guide, support information, a reference to the code examples and the video files. As you may notice I said “reference to the code examples”. Even if the name of the file “Code Bundle.zip” it contains only a file with the link to the source code on github.

Continue reading Review: Getting Started with Apache Maven by Russell Gold

Update Maven pom version on GIT checkout in TeamCity

Here is a solution to the following problems

  • Deriving #Maven artifact version from #GIT branch,
  • Update pom version on #GIT checkout automatically,
  • Add the ability to use Pull request with Apache #Maven.

You have a workflow requirement that require you to have the artifact version of a module externally defined from the current branch in #GIT.

For example

You want to start working on a new feature branch “feature-memory-improvement”, so you branch from master a new branch named feature/feature-memory-improvement

Having unique snapshot is a something you need to share your code using a #Maven repository, so you may want to have into the branch all pom.xml version changed to

<version>FEATURE-MEMORY-IMPROVEMENT-SNAPHOTS</version>

changing all your pom.xml and doing a technical commit  will create merge conflicts when using pull request!

One solution, while not perfect is to do the following:  You can add a separate execution to run a goal which will change the version of the POM automatically in the #Maven reactor. This small script will do it¨

Continue reading Update Maven pom version on GIT checkout in TeamCity

Tomcat 7 and Apache Maven

Here is 3 different way to control the lifetime a local Tomcat 7 container using Apache #Maven. A typical scenario would be to start a servlet container prior to running integration tests (Selenium, SAHI or using any other framework you can think of )

With the following examples, you will be able to start an instance of Tomcat 7 running your web application in the pre-integration-test phase and stop the instance in the post-integration-test phase. You can also decide to use an embedded container like Jetty instead.

Continue reading Tomcat 7 and Apache Maven