Mask sensitive data in logs with logback
00:49 13 Aug 2014

I need to be able to search an event for any one of a number of patterns and replace the text in the pattern with a masked value. This is a feature in our application intended to prevent sensitive information falling into the logs. As the information can be from a large variety of sources, it is not practical to apply filters on all the inputs. Besides there are uses for toString() beyond logging and I don't want toString() to uniformly mask for all calls (only logging).

I have tried using the %replace method in logback.xml:

%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %replace(%msg){'f k\="pin">(.*?)</f','f k\="pin">**********</f'}%n

This was successful (after replacing the angle brackets with character entities), but it can only replace a single pattern. I would also like to perform the equivalent of

%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %replace(%msg){'pin=(.*?),','pin=**********,'}%n

at the same time, but cannot. There is no way to mask two patterns in the one %replace.

The other way that has been loosely discussed on the interblags is extending something on the appender/encoder/layout hierarchy, but every attempt to intercept the ILoggingEvent has resulted in a collapse of the whole system, usually through instantiation errors or UnsupportedOperationException.

For example, I tried extending PatternLayout:

@Component("maskingPatternLayout")
public class MaskingPatternLayout extends PatternLayout {

    @Autowired
    private Environment env;

    @Override
    public String doLayout(ILoggingEvent event) {
        String message=super.doLayout(event);

        String patternsProperty = env.getProperty("bowdleriser.patterns");

        if( patternsProperty != null ) {
            String[] patterns = patternsProperty.split("|");
            for (int i = 0; i < patterns.length; i++ ) {
                Pattern pattern = Pattern.compile(patterns[i]);
                Matcher matcher = pattern.matcher(event.getMessage());
                matcher.replaceAll("*");
            }
        } else {
            System.out.println("Bowdleriser not cleaning! Naughty strings are getting through!");
        }
        
        return message;
    }
}

and then adjusting the logback.xml:


  
    
        
      %d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
        
    
  

    
      logs/touchpoint.log
        
            logs/touchpoint.%i.log.zip
            1
            3
        

        
            10MB
        
      
          
            %date{YYYY-MM-dd HH:mm:ss} %level [%thread] %logger{10} [%file:%line] %msg%n
          
      
    


  
  

  
    
    
  

I have tried many other insertions, so I was wondering if anyone has actually achieved what I am attempting and if they could provide any clues or a solution.

logback data-masking