Category Archives: Groovy

Groovy/Java: Handling Byte Order Marks When Reading a File

PROBLEM

Given a file with the following content:-

10,20

When reading the file:-

def inputStream = new FileInputStream('test.csv')
def value = inputStream.text.trim()

println "|${value}|"

… the following values are printed:-

| 10,20|

Even though the value is trimmed, there is still a leading space in front of text.

A further inspection reveals the leading space is not a regular space:-

// first character is not a space
assert value.charAt(0) != (char) ' '

// ASCII value: 65279 vs 32
assert (int) value.charAt(0) != (int) ((char) ' ').charValue()

SOLUTION

Some editors prepend a special Unicode character called a byte order mark (BOM) to the file.

The simplest way to remove this special character is to leverage Apache Commons IO’s BOMInputStream:-

def inputStream = new BOMInputStream(new FileInputStream('test.csv'))
def value = inputStream.text.trim()

println "|${value}|"

… and now, the values are printed correctly:-

|10,20|
Advertisements

Spring Security: Propagating Security Context to Spawned Threads

PROBLEM

Let’s assume we have the following Parent class…

@Service
class Parent {
    @Autowired
    Child child

    void run() {
        println "Parent: ${SecurityContextHolder.context.authentication?.principal}"

        child.run()

        println "Parent: Done"
    }
}

… and Child class…

@Service
class Child {
    @Async
    void run() {
        Thread.sleep(500)
        println "Child: ${SecurityContextHolder.context.authentication?.principal}"
    }
}

Let’s also assume the user has successfully logged in and Spring Security has set up the user authentication info.

The Parent will spawn a new thread (through @Async) to run Child.

When invoking the Parent, this is what we see:-

Parent: USER_PRINCIPAL
Parent: Done
Child: null

The Child, for some reason, doesn’t get the receive the user authentication info.

SOLUTION

By default, SecurityContextHolder uses MODE_THREADLOCAL to store the user authentication info. As a result, this info is not accessible to methods outside the current execution thread.

To fix this, configure SecurityContextHolder to use MODE_INHERITABLETHREADLOCAL to pass the user authentication info to other spawned threads.

@Configuration
@EnableAsync
class AppConfig {
    AppConfig() {
        SecurityContextHolder.setStrategyName(SecurityContextHolder.MODE_INHERITABLETHREADLOCAL)
    }
}

When invoking the Parent again, now the Child will also receive the user authentication object:-

Parent: USER_PRINCIPAL
Parent: Done
Child: USER_PRINCIPAL

JEE Security: Preventing Clickjacking Attacks

PROBLEM

Clickjacking is an attack that tricks the users to perform unintended actions… see OWASP’s Testing for Clickjacking (OTG-CLIENT-009)

SOLUTION

To prevent clickjacking attacks, the app must set X-FRAME-OPTIONS header with an appropriate value:-

  • DENY: this denies any domain using the page as an iFrame source. This is the best option.
  • SAMEORIGIN: this allows pages within the same domain to use other application pages as iFrame sources.
  • ALLOW-FROM [whitelisted domains]: this declares a list of domains that are allowed to include the pages as iFrame sources.

If set correctly, the HTTPS response should show X-FRAME-OPTIONS header:-

➜  ~ curl -i -k https://localhost:8443/
HTTP/1.1 200
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Frame-Options: DENY
X-Application-Context: application:local:8443
Set-Cookie: JSESSIONID=04ADDAF886A20AA561021E869E980BCC; Path=/; Secure; HttpOnly
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Content-Length: 631
Date: Thu, 31 Aug 2017 14:56:57 GMT

There are several ways to set this header.

Solution 1: Using a servlet filter

You may create a servlet filter that sets X-FRAME-OPTIONS in the response header.

Here’s an example using web.xml-less Spring Boot:-

@SpringBootApplication
class Application extends SpringBootServletInitializer {
    static void main(String[] args) {
        SpringApplication.run(Application, args)
    }

    @Override
    protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
        return builder.sources(Application)
    }

    @Bean
    FilterRegistrationBean clickjackingPreventionFilter() {
        return new FilterRegistrationBean(
                urlPatterns: ['/**'],
                filter: new Filter() {
                    @Override
                    void init(final FilterConfig filterConfig) throws ServletException {
                    }

                    @Override
                    void doFilter(final ServletRequest servletRequest,
                                  final ServletResponse servletResponse,
                                  final FilterChain filterChain) throws IOException, ServletException {
                        final HttpServletResponse response = (HttpServletResponse) servletResponse
                        response.addHeader('X-FRAME-OPTIONS', 'DENY')
                        filterChain.doFilter(servletRequest, servletResponse)
                    }

                    @Override
                    void destroy() {
                    }
                }
        )
    }
}

Solution 2: Using Spring Security

Spring Security provides a very easy way to set the X-FRAME-OPTIONS header:-

@Configuration
@EnableWebSecurity
class SecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(final HttpSecurity http) throws Exception {
        http.
                headers().frameOptions().deny(). 
                and().
                authorizeRequests().
                antMatchers('/**').permitAll()
    }
}

JEE Security: Disabling HTTP OPTIONS method

PROBLEM

HTTP OPTIONS method is used to provide a list of methods that are supported by the web server.

For example, the following shows both GET and HEAD are allowed on the given link:-

➜  ~ curl -i -k -X OPTIONS https://localhost:8443/
HTTP/1.1 200
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Frame-Options: DENY
X-Application-Context: application:local:8443
Allow: GET,HEAD
Content-Length: 0
Date: Thu, 31 Aug 2017 14:07:21 GMT

Enabling OPTIONS may increase the risk of cross-site tracing (XST)… see OWASP’s Test HTTP Methods (OTG-CONFIG-006).

SOLUTION

There are several ways to disable OPTIONS method.

Solution 1: Using web.xml

If your app has web.xml, you may add the following snippet:-

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
		 http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         metadata-complete="true" version="3.1">

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>restricted methods</web-resource-name>
            <url-pattern>/*</url-pattern>
            <http-method>OPTIONS</http-method>
        </web-resource-collection>
        <auth-constraint/>
    </security-constraint>
	
    <!-- Other configurations -->
</web-app>

Solution 2: Using Spring Boot

If you are using Spring Boot, there isn’t any option to mimic the above configuration programmatically.

However, you still can use web.xml in conjunction with Spring Boot by setting metadata-complete to false and use servlet version 3.0 or higher:-

<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://xmlns.jcp.org/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://xmlns.jcp.org/xml/ns/javaee
		 http://xmlns.jcp.org/xml/ns/javaee/web-app_3_1.xsd"
         metadata-complete="false" version="3.1">

    <security-constraint>
        <web-resource-collection>
            <web-resource-name>restricted methods</web-resource-name>
            <url-pattern>/*</url-pattern>
            <http-method>OPTIONS</http-method>
        </web-resource-collection>
        <auth-constraint/>
    </security-constraint>
</web-app>

Solution 3: Using Spring Security

If you don’t want to use web.xml, you may configure Spring Security to disable OPTIONS method on all URIs:-

@Configuration
@EnableWebSecurity
class SecurityConfig extends WebSecurityConfigurerAdapter {
    @Override
    protected void configure(final HttpSecurity http) throws Exception {
        http.authorizeRequests().
                antMatchers(HttpMethod.OPTIONS, '/**').denyAll().
                antMatchers('/**').permitAll()
    }
}

Now, when trying to hit the same link with OPTIONS method, the app will return 403 Forbidden:-

➜  ~ curl -i -k -X OPTIONS https://localhost:8443/
HTTP/1.1 403
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: 0
Strict-Transport-Security: max-age=31536000 ; includeSubDomains
X-Frame-Options: DENY
Content-Type: application/json;charset=UTF-8
Transfer-Encoding: chunked
Date: Thu, 31 Aug 2017 14:26:51 GMT

Groovy: Copying Properties Between Two Beans

PROBLEM

Given two beans…

class A {
    String name
    LocalDateTime localDateTime
}

class B {
    String name
    LocalDateTime localDateTime
}

There are several ways to copy properties from one bean to another:-

  • The most rudimentary way is to “get” each property from one bean and “set” it on another bean, which is VERY verbose and stupid.
  • Another way is to leverage utilities such as BeanUtils provided by either Apache Commons or Spring. While both libraries are called BeanUtils, they behave slightly different from one another.
  • Write home-grown reflection function… and now you have two problems: 1) it may not handle edge cases properly and 2) no one understands your implementation.

SOLUTION

Groovy provides a helper class to solve this problem called InvokerHelper. The advantage of using this is there’s no need to import yet another dependency and it still allows us to keep our code concise.

Scenario 1: Both beans have exact properties

class MySpec extends Specification {
    class A {
        String name
        LocalDateTime localDateTime
    }

    class B {
        String name
        LocalDateTime localDateTime
    }

    def "given a and b with same exact properties, should copy all properties"() {
        given:
        def a = new A(name: 'name',
                      localDateTime: LocalDateTime.now())
        def b = new B()

        when:
        InvokerHelper.setProperties(b, a.properties)

        then:
        b.name == a.name
        b.localDateTime == a.localDateTime
    }
}

Scenario 2: Source bean has additional properties

class MySpec extends Specification {
    class A {
        String name
        LocalDateTime localDateTime
        Integer extra1
        Boolean extra2
    }

    class B {
        String name
        LocalDateTime localDateTime
    }

    def "given a has additional properties than b, should ignore additional properties"() {
        given:
        def a = new A(name: 'name',
                      localDateTime: LocalDateTime.now(),
                      extra1: 1,
                      extra2: true)
        def b = new B()

        when:
        InvokerHelper.setProperties(b, a.properties)

        then:
        b.name == a.name
        b.localDateTime == a.localDateTime
    }
}

Scenario 3: Destination bean has additional properties

class MySpec extends Specification {
    class A {
        String name
        LocalDateTime localDateTime
    }

    class B {
        String name
        LocalDateTime localDateTime
        Integer extra1
        Boolean extra2
    }

    def "given b has additional properties than a, should set additional properties as null"() {
        given:
        def a = new A(name: 'name',
                      localDateTime: LocalDateTime.now())
        def b = new B()

        when:
        InvokerHelper.setProperties(b, a.properties)

        then:
        b.name == a.name
        b.localDateTime == a.localDateTime
        b.extra1 == null
        b.extra2 == null
    }
}

Scenario 4: Same property but different data type from each bean

The short answer is don’t do it. It’s not worth the hassle and confusion.

class MySpec extends Specification {
    class A {
        String number
    }

    class B {
        Integer number
    }

    def "given same property name but different data type, should go bat shit crazy"() {
        given:
        def a = new A(number: '0')
        def b = new B()

        when:
        InvokerHelper.setProperties(b, a.properties)

        then:
        b.number == 48 // ASCII value for character '0'
    }

    def "given same property name but different data type, should go bat shit crazy again"() {
        given:
        def a = new A(number: '10')
        def b = new B()

        when:
        InvokerHelper.setProperties(b, a.properties)

        then:
        thrown ClassCastException // because there's no ASCII value for character '10'
    }
}

Java: Exploring Preferences API

BACKGROUND

In any written scripts or rich client apps, there is almost a need to persist the user preferences or app configurations.

Most of the time, we, the proud developers, handle that situation in very ad-hoc manner. When storing in a file, we use different formats: from old-boring XML, to cool-kid JSON, to even cooler-kid YAML or the kindergarten-kid key=value property. Then, we have to decide where to write the file to, whether to use C:\ and screw your non-windows users, whether to use backslashes to construct the file path or forward slashes because we are sick and tired escaping the effing backslashes.

The long story short is… yes, we, the proud developers, can do all that… or, as one of my current project peers like to say, “make it configurable” on literally everything to the point it’s pretty close of becoming a drinking game now.

But, the point I want to make here is… we are consistent on being inconsistent.

SOLUTION

Java provides the Preferences API as an attempt to solve this mess. Using this API, the developers do not need to know where or how to store the user preferences or app configurations. Rather, it relies on the native API to store the data: registry for Windows, .plist for Mac and XML for Unix/Linux.

The most interesting part is… the Preferences API has been around since JDK 1.4.

Code wise, it doesn’t get any simpler than this:-

// create new configuration or reference existing configuration
Preferences preferences = Preferences.userNodeForPackage(WuTangClan)

// insert/update 3 key/value pairs
preferences.put('key1', 'value1')
preferences.put('key2', 'value2')
preferences.put('key3', 'value3')

// returns 'value2'
println preferences.get('key2', '-')

// returns '-'
println preferences.get('invalid', '-')

// remove by key
preferences.remove('key3')

// delete everything
preferences.removeNode()

But, where and how exactly do Mac and Windows store this data?

There are several ways to get an instance of Preferences.

Preferences.userNodeForPackage(WuTangClan)

Mac

If WuTangClan class file is located under wu.tang.clan.config package, the configuration file is created at ~/Library/Preferences/wu.tang.clan.plist with the following content:-

{    "/wu/tang/clan/" = {
        "config/" = {
            "key1" = "value1";
            "key2" = "value2";
        };
    };
}

64-bit Windows + 64-bit JVM

Configuration is stored in the registry with the following key:-

HKEY_CURRENT_USER\SOFTWARE\JavaSoft\Prefs\wu\tang\clan\config

Preferences.userRoot().node(‘path’)

Example 1

Let’s assume we have this:-

Preferences.userRoot().node('wu')

Mac

The configuration is created at ~/Library/Preferences/com.apple.java.util.prefs.plist with the following content:-

{    "/" = {
        ...
        
        "wu/" = {
            "key1" = "value1";
            "key2" = "value2";
        };
        
        ...
    };
}

This file also contains configurations from other installed software.

64-bit Windows + 64-bit JVM

Configuration is stored in the registry with the following key:-

HKEY_CURRENT_USER\SOFTWARE\JavaSoft\Prefs\wu

Example 2

How about this?

Preferences.userRoot().node('wu/tang')

// ... OR ...

Preferences.userRoot().node('wu').node('tang')

Mac

The configuration still resides under ~/Library/Preferences/com.apple.java.util.prefs.plist with the following content:-

{    "/" = {
        ...
        "wu/" = {
            "tang/" = {
                "key1" = "value1";
                "key2" = "value2";
            };
        };
        ...
    };
}

64-bit Windows + 64-bit JVM

Configuration is stored in the registry with the following key:-

HKEY_CURRENT_USER\SOFTWARE\JavaSoft\Prefs\wu\tang

Example 3

How about this?

Preferences.userRoot().node('wu/tang/clan')

// ... OR ...

Preferences.userRoot().node('wu').node('tang').node('clan')

Mac

Now, the shit is about to get real here.

Mac, for some reason, creates a stub under ~/Library/Preferences/com.apple.java.util.prefs.plist with the following content:-

{    "/" = {
        ...
        "wu/" = { "tang/" = { "clan/" = { }; }; };
        ...
    };
}

The actual configuration now resides under ~/Library/Preferences/wu.tang.clan.plist:-

{    "/wu/tang/clan/" = {
        "key1" = "value1";
        "key2" = "value2";
    };
}

It appears when the path reaches certain depth, Mac will create the separate configuration file for it.

64-bit Windows + 64-bit JVM

Configuration is stored in the registry with the following key:-

HKEY_CURRENT_USER\SOFTWARE\JavaSoft\Prefs\wu\tang\clan

Preferences.systemNodeForPackage(WuTangClan) or Preferences.systemRoot().node(‘path’)

Instead of storing the configuration at user level, we may also store the configuration at system level.

Mac

Instead of storing under ~/Library/Preferences, the configuration is stored under /Library/Preferences.

On top of that, based on Java Development Guide for Mac, the configuration is only persisted if the user is an administrator.

The really weird part is the code will not throw any exceptions due to insufficient permission.

64-bit Windows + 64-bit JVM

Instead of storing under HKEY_CURRENT_USER\[path], the configuration is stored under HKEY_LOCAL_MACHINE\[path].

Best Practices

I’m not sure if this is a best practice, but my personal preference is to specify my own string path through Preferences.userRoot().node(..).

Preferences.userNodeForPackage(..) worries me because if I refactor my code by moving the class files around, it may not find the existing configuration due to changed path.

When specifying the string path, do make sure the path value is rather unique to prevent reading an existing configuration from other installed software.

Spring + Ehcache: XML-less Spring Configuration for Ehcache 2.x vs Ehcache 3.x

BACKGROUND

The documentation on the web regarding Ehcache 3.x configuration using Spring is rather lacking. There is apparently a very distinct difference in Spring Java-based configuration between Ehcache 2.x vs Ehcache 3.x.

Spring + Ehcache 2.x

Dependency:-

<dependency>
    <groupId>net.sf.ehcache</groupId>
    <artifactId>ehcache</artifactId>
    <version>2.10.3</version>
</dependency>

Spring configuration:-

@Configuration
@EnableCaching
class Config {
    @Bean
    CacheManager cacheManager() {
        return new EhCacheCacheManager(ehCacheManager())
    }

    @Bean(destroyMethod = 'shutdown')
    net.sf.ehcache.CacheManager ehCacheManager() {
        CacheConfiguration cacheConfiguration = new CacheConfiguration(
                name: 'person',
                maxEntriesLocalHeap: 5,
                timeToLiveSeconds: 5
        )

        net.sf.ehcache.config.Configuration config = new net.sf.ehcache.config.Configuration()
        config.addCache(cacheConfiguration)

        return new net.sf.ehcache.CacheManager(config)
    }
}

Spring + Ehcache 3.x

Dependency:-

<dependency>
    <groupId>org.ehcache</groupId>
    <artifactId>ehcache</artifactId>
    <version>3.3.1</version>
</dependency>
<dependency>
    <groupId>javax.cache</groupId>
    <artifactId>cache-api</artifactId>
    <version>1.0.0</version>
</dependency>

Spring configuration:-

import org.ehcache.config.CacheConfiguration
import org.ehcache.config.builders.CacheConfigurationBuilder
import org.ehcache.config.builders.ResourcePoolsBuilder
import org.ehcache.core.config.DefaultConfiguration
import org.ehcache.expiry.Duration
import org.ehcache.expiry.Expirations
import org.ehcache.jsr107.EhcacheCachingProvider
import org.springframework.cache.annotation.EnableCaching
import org.springframework.cache.jcache.JCacheCacheManager
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration

import javax.cache.CacheManager
import javax.cache.Caching
import java.util.concurrent.TimeUnit

@Configuration
@EnableCaching
class Config {
    @Bean
    JCacheCacheManager jCacheCacheManager() {
        return new JCacheCacheManager(cacheManager())
    }

    @Bean(destroyMethod = 'close')
    CacheManager cacheManager() {
        CacheConfiguration cacheConfiguration = CacheConfigurationBuilder.newCacheConfigurationBuilder(
                Object,
                Object,
                ResourcePoolsBuilder.heap(5)).
                withExpiry(Expirations.timeToLiveExpiration(new Duration(5, TimeUnit.SECONDS))).
                build()

        Map<String, CacheConfiguration> caches = ['person': cacheConfiguration]

        EhcacheCachingProvider provider = (EhcacheCachingProvider) Caching.getCachingProvider()
        DefaultConfiguration configuration = new DefaultConfiguration(caches, provider.getDefaultClassLoader())

        return provider.getCacheManager(provider.getDefaultURI(), configuration)
    }
}