A previous blog post described the Java Gotcha "Bitwise vs Boolean Operator".

• Java Gotchas - Bitwise vs Boolean Operator

We added a variant of this, and some other Java Gotchas into a fun little Quiz called "Challenge The Sensei".

• scw.typeform.com/to/rgw7q7OE

If you've read the blog post above then you'll be in a good place to ace at least one of the questions.

But your friends might not, so if you find the quiz fun, you can share it with them and see if they score as well as you.

Since we don't want to just quiz you. We want to try and use this to help educate and codify the knowledge. So we have created a Github repo that has runnable code examples for the problem, and the solution.

• github.com/SecureCodeWarrior/challenge-the-sensei

This is a Sensei enabled repo.

When you clone the repo and load it into IntelliJ, assuming you have the Secure Code Warrior Sensei IntelliJ Plugin installed, then it will automatically see that you have a .sensei folder, and load the Sensei recipes.

When browsing the code in the IDE you should see IntelliJ prompt you that the error exists in the code, and this should make it easier to see the gotcha in the code:

• Hover over the highlighted code then you'll see a prompt telling you about the error
• Use the Show Context Action key: alt+enter (Windows) option+enter (macOS) and we may have a QuickFix available that can fix the code.

Sensei recipes have been added for:

• Bitwise operators
• Split IP Address
• Assert Order

We are adding more recipes and more explanatory text to cover the rest of the code in the future... but don't let that stop you from having a look at the code and spotting the error yourself.

And remember to try the quiz and "Challenge the Sensei"

Java Gotchas - Bitwise vs Boolean Operators

> "Java Gotcha" - a common mistake pattern that is easy to accidentally implement.

A fairly simple Java Gotcha to accidentally fall into is: using a Bitwise operator instead of a Boolean Comparison operator.

e.g. a simple mistype can result in writing "&" when you really meant to write "&&".

A common heuristic we learn when reviewing code is:

> "&" or "|" when used in a conditional statement is probably not intended.

In this blog post, we will explore the heuristic and identify ways we can identify and fix this coding issue.

What's the Problem? Bitwise operations work fine with Booleans

Using Bitwise operators with Booleans is perfectly valid, so Java will not report a syntax error.

If I construct a JUnit Test to explore a truth table for both Bitwise OR (|) and Bitwise AND (&) then we will see that the outputs from the Bitwise operator match the truth table. Given this, we might think that the use of Bitwise operators is not an issue.

AND Truth Table

@Test
    void bitwiseOperatorsAndTruthTable(){
          Assertions.assertEquals(true, true & true);
          Assertions.assertEquals(false, true & false);
          Assertions.assertEquals(false, false & true);
          Assertions.assertEquals(false, false & false);
    }

The test passes, this is perfectly valid Java.

OR Truth Table

   @Test
    void bitwiseOperatorsOrTruthTable(){
        Assertions.assertEquals(true, true | true);
        Assertions.assertEquals(true, true | false);
        Assertions.assertEquals(true, false | true);
        Assertions.assertEquals(false, false | false);
    }

This test also passes, why do we prefer "&&'and "||'?

Truth table images were created using the truth table tool from web.standfor.edu.

Issue: Short Circuit Operation

The real issue is the difference in behaviour between Bitwise (&, |) and Boolean (&&, ||) operators.

A Boolean operator is a short circuit operator and only evaluates as much as it needs to.

e.g.

if (args != null & args.length() > 23) {
    System.out.println(args);
}

In the above code, both boolean conditions will evaluate, because the Bitwise operator has been used:

• args != null
• args.length() > 23

This leaves my code open to a NullPointerException if args is null because we will always perform the check for args.length, even when args is null because both boolean conditions have to be evaluated.

Boolean Operators Short Circuit Evaluation

When an && is used e.g.

if (args != null && args.length() > 23) {
    System.out.println(args);
}

As soon as we know that args != null evaluates to false the condition expression evaluation stops.

We don't need to evaluate the right-hand side.

Whatever the outcome of the right-hand side condition, the final value of the Boolean expression will be false.

But this would never happen in production code

This is a pretty easy mistake to make and is not always picked up by Static Analysis tools.

I used the following Google Dork to see if I could find any public examples of this pattern:

filetype:java if "!=null & "
This search brought back some code from Android in the RootWindowContainer
isDocument = intent != null & intent.isDocument()

This is the type of code that might pass a code review because we often do use Bitwise operators in assignment statements to mask values. But in this instance, the outcome is the same as the if statement example above. If intent is ever null, then a NullPointerException will be thrown.

Very often we get away with this construct because we often code defensively and write redundant code. The check for != null may well be redundant in most use cases.

This is an error made by programmers in production code.

I don't know how current the results for the search are, but when I ran the search there were results back with code from: Google, Amazon, Apache... and me.

A recent pull request on one of my open source projects was to address exactly this error.

if(type!=null & type.trim().length()>0){
    acceptMediaTypeDefinitionsList.add(type.trim());
}

How to Find This

When I checked my sample code in a few static analysers, none of them picked up this hidden self-destruct code.

As a team at Secure Code Warrior, we created and reviewed a fairly simple Sensei recipe that could pick this up.

Because Bitwise operators are perfectly valid, and often used in assignments we focussed on the use-case of if statements, and the use of Bitwise &, to find the problematic code.

search:
  expression:
    anyOf:
    - in:
        condition: {}
    value:
      caseSensitive: false
      matches: ".* & .*"

This uses a regular expression to match " & " when it is used as a condition expression. e.g. in an if statement.

To fix it, we again relied on regular expressions. This time using the sed function in the QuickFix to globally replace the & in the expression with &&.

availableFixes:
  - name: "Replace bitwise AND operator to logical AND operator"
    actions:
      - rewrite:
          to: "{{#sed}}s/&/&&/g,{{{ . }}}{{/sed}}"

End Notes

This covers the most common misuse of a Bitwise operator, i.e. when a Boolean operator was actually intended.

There are other situations where this could crop up e.g. the assignment example, but when writing recipes we have to attempt to avoid false-positive identification, otherwise recipes will be ignored or turned off. We build recipes to match the most common occurrences. As Sensei evolves, we may well add additional specificity into the search functionality to cover more matching conditions.

In its current form, this recipe would identify many of the live use-cases, and most importantly, the one that was reported in my project.

NOTE: A fair few code warriors contributed to this example and recipe review - Charlie Eriksen, Matthieu Calie, Robin Claerhaut, Brysen Ackx, Nathan Desmet, Downey Robersscheuten. Thanks for your help.

---

You can install Sensei from within IntelliJ using "Preferences \ Plugins" (Mac) or "Settings \ Plugins" (Windows) then just search for "sensei secure code"

We have a lot of source code and recipes for these blog posts (including this one) in the `sensei-blog-examples` repository in the Secure Code Warrior GitHub account.

https://github.com/securecodewarrior/sensei-blog-examples

Learn more about Sensei

What is Static Analysis?

Static Analysis is the automated analysis of source code without executing the application.

When the analysis is performed during program execution then it is known as Dynamic Analysis.

Static Analysis is often used to detect:

• Security vulnerabilities.
• Performance issues.
• Non-compliance with standards.
• Use of out of date programming constructs.

How does a Static Analysis tool work?

The basic concept common to all Static Analysis tools is searching source code to identify specific coding patterns that have some sort of warning or information associated with them.

This could be as simple as "JUnit 5 test classes do not need to be 'public'". Or something complex to identify like "Untrusted String input being used in an SQL execution statement".

Static Analysis tools vary in how they implement this functionality.

• source code parsing technology to create an Abstract Syntax Tree (AST),
• text Regular Expression matching,
• a combination of the above.

Regular Expression matching on text is very flexible, easy to write rules to match, but can often lead to a lot of false positives and the matching rules are ignorant of the surrounding code context.

AST matching treats the source code as program code, and not just files filled with text, this allows for more specific, contextual matching and can reduce the number of false positives reported against the code.

Static Analysis in Continuous Integration

Static Analysis is often performed during the Continous Integration (CI) process to generate a report of compliance issues which can be reviewed to receive an objective view of the code-base over time.

Some people use Static Analysis as an objective measure of their code quality by configuring the static analysis tool to only measure specific parts of the code, and only report on a subset of rules.

The objectivity is provided by the rules used since these do not vary in their evaluation of code over time. Clearly, the combination of rules used and their configuration is a subjective decision and different teams choose to use different rules at different times.

Having the Static Analysis performed in CI is useful but might delay the feedback to the programmer. Programmers don't receive feedback when coding, they receive feedback later when the code is run through the Static Analysis tool. Another side-effect of running the Static Analysis in CI is that the results are easier to ignore.

To help make teams pay more attention to the results from Static Analysis it is usually possible to configure a threshold metric in the build process to fail the build if the metric is exceeded e.g. a number of rules triggered.

Static Analysis in the IDE

To receive feedback faster, there are many IDE plugins that run the Static Analysis rules in the IDE on demand, or periodically as the code changes.

The rule violations can then be seen in the IDE as the programmer is writing code, and to make the rules harder to ignore, the violations can often be configured to render as underlined code in the editor.

I personally find this a useful way to improve my coding, particularly when working with a new library that is covered by the Static Analysis tool. Although it can be 'noisy' with false positives, or rules you are not interested in. But this is solved by taking the extra step to configure the Static Analysis tool to ignore certain rules.

Fixing Code Based on Static Analysis Rules

With most Static Analysis tools, the fixing of the rule is left to the programmer, so they have to understand the cause of the rule violation and how to fix it.

Very few static analysis tools also include the ability to fix the violations because the fix is so often contextual to the team and the technology used and their agreed coding styles.

Default Rules

False confidence in the quality of the rules may arise when the Static Analysis tools come with default rules, it is tempting to believe that they cover all the issues that a programmer might encounter, and all the circumstances that rule should apply. Sometimes the circumstances in which a rule should apply can be subtle and may not be easy to detect.

The hope is that by using a Static Analysis tool, and researching the rules and violations in more detail, that programmers will develop the skill to detect and avoid the issue in the context of their specific domain.

When the domain requires contextual rules, the Static Analysis tools may not have any rules that match your domain or library, and additionally, the tools can often be difficult to configure and expand.

Annoyances

None of these 'annoyances' are insurmountable:

• false positives
• lack of fixes
• configuration to ignore rules
• adding context-specific rules

But they are often used as excuses to avoid using Static Analysis tools in the first place, which is a pity because the use of Static Analysis can be enormously useful, as a way to:

• highlight better approaches to junior developers
• gain fast feedback on clear coding violations
• identify obscure issues that the programmer has not encountered before
• reinforce that the programmer has adopted a good coding approach (when no violations are reported)

IDE Based Static Analysis Tools

As an individual contributor to a project, I like to use Static Analysis tools that run from within the IDE so that I receive fast feedback on my code.

This supplements any pull request review process, and CI integration that a project may have.

I try to identify tools that will give me an edge, and improve my individual workflow.

When tools run in the IDE, because they tend to share the same basic GUI and configuration approach, it can be tempting to view them interchangeably.

The tools may have overlapping functionality or rule sets but to gain maximum advantage I install multiple tools to take advantage of their strengths.

The Static Analysis IDE tools I actively use when coding are listed below:

• the inbuilt IntelliJ Inspections - common coding patterns
• SpotBugs - common errors
• SonarLint - common usage patterns
• CheckStyle - common style patterns
• Sensei from Secure Code Warrior - custom rule creation

I use them all because they work well together to augment and supplement each other.

IntelliJ Inspections

If you use IntelliJ then you are already using their Inspections.

These are Static Analysis rules which are flagged in the IDE. Some of them also have QuickFix options to rewrite the code to address the issue.

The rules are configurable on and off, and to choose the error level used to highlight it in the IDE.

There are a lot of good IntelliJ Inspections. I know that because I read through them while writing this. I use the IntelliJ Inspections as the defaults and haven't configured them, but to gain full value from the Inspections you should read through them, identify those relevant to your coding style, and configure the warning level so that you notice them in the code.

The great thing about the IntelliJ Inspections is that they come free with the IDE and they help build the muscle memory of:

• noticing warnings and errors in the source as you write code
• hovering the mouse over flagged code to learn the rule violations
• using alt+enter to apply a QuickFix for the issue

SpotBugs

The SpotBugs IntelliJ plugin uses Static Analysis to try and alert you to bugs in your code.

SpotBugs can be configured from the IntelliJ Preferences to scan your code, the actual rules used can be found in the Detector tab.

I tend to use SpotBugs after I've written and reviewed my code, then I'll run the 'Analyze Project Files Including Test Sources'.

This does help me identify bugs, dead code, and obvious optimizations. It also forces me to research some of the flagged violations to help me decide what to do.

SpotBugs will find issues but does not offer any QuickFix actions to attempt to resolve the issues.

SpotBugs is easy to configure and I find it to be a useful objective second opinion to consult in my IDE.

SonarLint

The SonarLint plugin.

SonarLint can be configured from the IntelliJ Preferences to select which rules the code is validated against.

By default, SonarLint runs in realtime and shows issues for the current code that you are editing.

SonarLint does not offer quick fixes but the documentation associated with the violation reports is usually clear and well documented.

I've found SonarLint to be useful in the past for alerting me to new Java features that I was aware of in the newer versions of Java.

CheckStyle

The CheckStyle plugin offers a mix of formatting and code-quality rules.

The CheckStyle plugin comes bundled with 'Sun Checks' and 'Google Checks'.

The definitions of these can be easily found online.

CheckStyle adds the most value when a project has spent the time creating its own ruleset. Then the IDE plugin can be configured to use that ruleset and programmers can perform a scan, prior to committing the code to CI.

CheckStyle is very often used as a build failing plugin for CI processes when the number of CheckStyle violations exceeds a threshold.

Sensei

Sensei uses Static Analysis based on an Abstract Syntax Tree (AST) for matching code and for creating QuickFixes, this allows for very specific identification of code with issues.

The AST allows QuickFixes associated with a recipe to understand the surrounding code e.g. when adding a new class into the code, any import for that class will only be added to the source file once, and not for each replacement.

Sensei was created to make it easy to build custom matching rules which may not exist, or which would be hard to configure, in other tools. 

Rather than amend a configuration file, all the configuration can be performed in the GUI. When creating new recipes the GUI makes it easy to see which code the recipe matches. And when defining the QuickFixes the before and after state of the code can be compared immediately. This makes it easier to create very contextual recipes i.e. unique to teams, or technology, and even individual programmers.

I use Sensei in combination with other Static Analysis tools e.g. most Static Analysis tools will find issues, but not fix them. A common use case for Sensei is to replicate the other tool's matching search in Sensei, and expand it with a Quick Fix. This has the benefit that the custom fix applied already meets the coding standards for your project.

I periodically find myself creating Sensei recipes that already exist in the IntelliJ Intensions set because the Intension report doesn't quite match the context I've created or because the QuickFix provided by IntelliJ doesn't match the code pattern I want to use.

I augment the existing tools, rather than attempt to fully replace them.

Sensei can also be very useful when you identify a contextual variant of a common rule e.g. if you are using an SQL library not supported by the Static Analysis tool, but the common SQL rules in the Static Analysis engine still apply, then you can create library-specific variants of those rules using Sensei.

Sensei does not come out of the box with a lot of generic recipes like the Static Analysis tools mentioned, its strength is in making it easy to create new recipes, complete with QuickFixes configured to match your specific coding style and use-cases.

NOTE: we are working on a public repository of recipes to cover generic use-cases, and you can find it here.

Summary

I tend to pick tools that work together, are configurable, and easy to expand to meet my specific context. I've used Static Analysis tools in the IDE for years to help me identify issues, and learn more about the programming languages and libraries I use.

I use all of the tools mentioned in combination:

• IntelliJ Intentions helps flag common code issues out the box, often with associated QuickFixes.
• SpotBugs finds simple bugs I might have missed and alerts me to performance issues.
• SonarLint identifies Java features I was unaware of and prompts me to additional ways of modelling my code.
• CheckStyle helps me conform to an agreed coding style that is also enforced during CI.
• Sensei helps me create QuickFixes to augment common scenarios found by Static Analysis tools and create specific project or technology recipes that can be hard to configure in another tool.

---

Install Sensei from within IntelliJ using "Preferences \ Plugins" (Mac) or "Settings \ Plugins" (Windows) then just search for "sensei secure code".

You can find a repository of example code and recipes for common use-cases in the Secure Code Warrior GitHub account, in the `sensei-blog-examples` project.

https://github.com/securecodewarrior/sensei-blog-examples

Learn more about Sensei

The Sensei project itself has its own set of recipes that have built up over time. This blog post is an example of one of the scenarios the Sensei team built a recipe for. A misconfiguration of Guice, which led to a NullPointerException being reported at runtime during testing.

This can be generalized to many Dependency Injection scenarios where code is syntactically correct, but because the wiring configuration was incorrect, an error slips through.

This often happens when we are learning the technology, and we keep making the simple mistake of forgetting to wire things up. But this also happens to experienced professionals because, well... we all make mistakes, and we may not have Unit Tests to cover everything.

RunTime Exceptions from Incorrect Dependency Injection Wiring

The code below fails at runtime with a NullPointerException.

‍

injector = Guice.createInjector(new SystemOutModule());
CountReporter reporter = injector.getInstance(CountReporter.class);
String [] lines5 = {"1: line", "2: line", "3: line", "4: line", "5: line"};
reporter.reportThisMany(Arrays.asList(lines5));
Assertions.assertEquals(5, reporter.getCount());

The code is syntactically correct but fails because we missed out a requestStaticInjection in our SystemOutModule configuration.

‍

public class SystemOutModule extends AbstractModule {
    @Override
    protected void configure() {
        binder().bind(ILineReporter.class).to(SystemOutReporter.class);
    }
}

When we try to use the reporter, created using the Injector, it is not fully instantiated and we receive a NullPointerException when we call reportThisMany.

We may well have missed that in our code review, or we didn't have unit tests that triggered the dependency injection, and it slipped out into our build.

‍

Warning Signs

In this case, there is a warning sign, the CountReporter has a static field annotated with @Inject but... the CountReporter class itself is package private.

In a complicated code base this could be a warning sign that the code is incorrect because the Module class configuring the bindings needs to be in the same package for this to work.

‍

class CountReporter {
    @Inject
    private static ILineReporter reporter;

Another error that we made, which we might have picked up in a code review, is that we forgot to actually bind the fields in the SystemOutModule configure method.

binder().requestStaticInjection(CountReporter.class);

Had we written the requestStaticInjection code then the Syntax Error generated when trying to use the CountReporter would have alerted us to the simple error.

> 'reporters.CountReporter' is not public in 'reporters'. Cannot be accessed from outside package

‍

Sadly. We forgot, and there were no syntactic warning signs in the code.

How could Sensei help?

We probably wouldn't use Sensei to pick up the missing requestStaticInjection since all our Guice configuration wiring would need to use that method, and we can't guarantee that all wiring is going to be as simple as this use-case.

We could write a Sensei rule to look for some warning signs that our code is not up to scratch.

In this case that would mean:

• Find any classes with @Inject annotated fields
• Where the classes are not public.

The above was the warning sign that they were unlikely to have been wired up.

By creating a recipe, then we will have a warning sign early, during the coding, and reduce the reliance on our pull requests or resolving our tech debt to allow us to add Unit Tests.

How to create a recipe?

The task I want to complete is:

• Create a recipe that matches fields annotated with @Inject which are in protected private classes

That should hopefully give us enough warning to identify any Modules using it and add the missing wiring code.

In my CountReporter class, I will use Alt+Enter to Create a new Recipe and I will start from scratch

I will name this and add a description:

Name: Guice: Injected Field Not Public
Description: If the Injected field is not public then the code might not be wired up
Level: Warning

The search I write looks for a class with a field annotated as Inject but which has not been scoped as public.

search:
  field:
    with:
      annotation:
        type: "com.google.inject.Inject"
    in:
      class:
        without:
          modifier: "public"

Fix

The QuickFix in the recipe will amend the injected class by changing the scope. But that is not the only code I have to change.

availableFixes:

- name: "Change class to public. Remember to also request injection on this class"
  actions:
  - changeModifiers:
      visibility: "public"
      target: "parentClass"

When the recipe is triggered I still have a manual step to perform in my code, adding the line containing requestStaticInjection to fully instantiate the object.

public class SystemOutModule extends AbstractModule {
    @Override
    protected void configure() {
        binder().bind(ILineReporter.class).to(SystemOutReporter.class);
        // instantiate via dependency injection
        binder().requestStaticInjection(CountReporter.class);
    }
}

I could potentially write another recipe to pick this up. I probably wouldn't do that unless forgetting to add the static injection became a semi-regular error that I made when coding.

‍

Summary

If we ever find ourselves making a mistake with a common root pattern, then Sensei can help codify the knowledge around detecting and fixing the issue, and then hopefully, it won't slip through code reviews and into production.

Sometimes the recipes we write identify heuristic patterns i.e. matching them doesn't guarantee that there is a problem, but is likely that there is a problem.

Also, the recipes and QuickFixes that we write, don't have to be fully comprehensive, they need to be good enough that they help us identify and fix problems without being overcomplicated. Because when they become overcomplicated they become harder to understand and harder to maintain.

---

You can install Sensei from within IntelliJ using "Preferences \ Plugins" (Mac) or "Settings \ Plugins" (Windows) then just search for "sensei secure code".

The source code and recipes for this post can be found in the `sensei-blog-examples` repository in the Secure Code Warrior GitHub account, in the `guiceexamples` module.

https://github.com/securecodewarrior/sensei-blog-examples

Learn more about Sensei

Amending Method and Class Visibility for JUnit 5

One of the joys of programming is the constant learning required to keep up to date. One of the issues is that we build up familiarity and patterns of usage that can impact the adoption of new approaches. Sensei can help migration by identifying deprecated patterns and prompting us with the fix to use going forward.

As an example, when I migrated from JUnit 4 to JUnit 5, I was used to writing all my test classes and methods as public. But with JUnit 5 they can be package private.

e.g. instead of:

public class Junit5VisibilityTest {
    @Test
    public void thisDoesNotNeedToBePublic(){
        Assertions.assertTrue(true);
    }
}

I really want to write:

class Junit5VisibilityTest {
    @Test
    void thisDoesNotNeedToBePublic(){
        Assertions.assertTrue(true);
    }
}

It took me a while to build the muscle memory to code to this, and I still slip up once in a while.

Using Sensei

With Sensei I can create recipes that find the public methods and classes, and amend the declarations to be package private automatically.

To achieve this I created a recipe:

Name - JUnit: JUnit 5 test methods do not need to be public
Description - JUnit 5 test methods do not need public visibility
Level - Error

I classed it as Error because I want to stamp out this coding practice and I want higher visibility of the issue when I'm writing code in the IDE.

Amending the Class Declaration

To find the classes, I search for any class which has a child annotation of @Test from Junit 5 i.e. org.junit.jupiter.api.Test

And where the class has modifier public:

search:
  class:
    with:
      child:
        annotation:
          type: "org.junit.jupiter.api.Test"
    modifier: "public"

Then the quick fix changes the modifier to remove the visibility so that it is the default, and the default is package private which is what I'm looking for.

availableFixes:
- name: "remove public visibility from JUnit 5 Test class"
  actions:
  - changeModifiers:
      visibility: ""

Amending the Method Declarations

The method declaration amendment recipe is much the same as the class recipe.

First I search for public methods annotated with @Test from JUnit 5.

search:
  method:
    annotation:
      type: "org.junit.jupiter.api.Test"
    modifier: "public"

And then I change the modifier to be default visibility.

availableFixes:
- name: "Remove @Test method public visibility"
  actions:
  - changeModifiers:
      visibility: ""

Hint: Amending Multiple Methods

Sensei has the ability to apply the QuickFix to all the violations in the current file.

When I use alt+enter to apply the QuickFix.

If I expand the QuickFix name menu, I can see an option to:

"Fix All: 'JUnit: JUnit 5 test methods do not need to be public' problems in the file"

When I select that option then Sensei will amend all the occurrences of the problem, not just the one I select.

Amending the class

In the same way that a method does not need to be public, neither does the class.

I can create a recipe and a QuckFix to amend the class.

Name - JUnit: Junit 5 Test classes do not need to be public
Description - Junit 5 Test classes do not need to be public
Level - Error

When I find a class that is public and has a method with a @Test annotation. Then I want to change the visibility.

search:
  class:
    modifier: "public"
    anyOf:
    - child:
        method:
          annotation:
            type: "Test"

I can make the change to the class definition with the changeModifiers action again.

availableFixes:
- name: "Remove @Test class public visibility"
  actions:  
- changeModifiers:
      visibility: ""

Summary

A static analysis tool initially alerted me to this recommended approach in JUnit. But the static analysis tool didn't help me build the muscle memory to change my code as I program.

Use the 'Level' to alert you. When it is a problem I am trying to stamp out in my coding I initially make it 'Error' and then reduce this as I wean myself off the coding approach.

Remember you can use Sensei to fix all the issues in the current file at the same time, by using the drop-down menu option when applying the QuickFix.

By creating a Sensei recipe, I can see my old coding approach in real-time. And QuickFix it, to reinforce the approach if I occasionally slip up in my coding.

---

You can install Sensei from within IntelliJ using "Preferences \ Plugins" (Mac) or "Settings \ Plugins" (Windows)  then just search for "sensei secure code".

The source code and recipes for this can be found in the `sensei-blog-examples` repository in the Secure Code Warrior GitHub account, in the `junitexamples` module.

• https://github.com/securecodewarrior/sensei-blog-examples

Automatically Adding a Private Constructor with Sensei

In a utility class, when the fields and methods are static,  there is no obvious reason why I would ever instantiate it.

e.g. UtilityClass utility = new UtilityClass();

The code below is a simple implementation of a Utility class.

public class UtilityClass {
    public static final Boolean ULTIMATE_TRUTH = true;
    public static boolean getTrue(){
        return ULTIMATE_TRUTH;
    }
}

This is the type of coding pattern that Static Analysis tools can pick up, but they often don't supply the ability to fix the issue.

I can use Sensei to identify the coding pattern, and automatically generate a private constructor to make it impossible for me to instantiate the class.

Searching for the Class

I will add a new recipe on the Utility class called:

•  Static Classes: create private constructor.

And initially, I'll create a simple matcher to search for a class.

search:

  class: {}

This will match any class, which is good enough to let me get started writing a Quick Fix. Once I have a Quick Fix that works, I'll refine the search to make it highlight when there is more likely to be a class that requires a private constructor.

Quick Fix

For the Quick Fix, I will want to generate a private constructor.

In the example class this would look like:

   private UtilityClass(){}

To add the above code to my class, my Quick Fix will add a Method, and the name of the method will be a Mustache template that uses the name of the class.

availableFixes:

- name: "add private constructor"
  actions:
  - addMethod:
      method: "private {{{ name }}}(){}"

In the GUI Editor, I use the Show Variables to create the Mustache template and then edit the field to add the private modifier, brackets, and braces to make it syntactically correct.

This would now allow me to add a private constructor to any class.

The QuickFix preview helps me because I can see the generated code as I write the Mustache template.

Now that I know I can fix the problem. I'll refine the search conditions to show the recipe when it is most appropriate.

Search for Missing Constructors

Ideally, I don't want to create a recipe that flags an error against every class. So I'll add some additional conditions in the search so that we only match on classes that do not have a constructor.

search:

  class:
    without:
      child:
        method:
          constructor: true

The YAML is slightly different from the GUI.

In the GUI I configure it to look for a class without a child method which is a constructor 'yes'. We use 'yes' in the GUI instead of 'true' to make the GUI a little more human friendly.

This recipe will now only reveal itself for any class without a constructor.

Narrow Search for Likely Culprits

So I might want to go further and look for the presence of static methods or fields.

I look for any class without a constructor and which has either all public static fields or all public static methods.

search:

  class:
    with:
      anyOf:
      - child:
          method:
            allOf:
            - modifier: "public"
            - modifier: "static"
      - child:
          field:
            allOf:
            - modifier: "static"
            - modifier: "public"
    without:
      child:
        method:
          constructor: true

Since Sensei is used to help me, as a  programmer, in the IDE, rather than to statically analyze the code and report all errors, this filter is good enough to rule out most classes in my code base where I might have a good reason to have a default public constructor.

In some projects, this might be a step too far because the utility classes could have private methods, so I might choose to look for the presence of 'any' public static methods, rather than 'all'.

       - child:
           field:
              anyOf:
                - modifier: "static"
                - modifier: "public"

Hints

Sensei is not designed to replace a Static Analysis tool. Sensei can augment a Static Analysis tool for common issues associated with your coding process, or technology. By replicating enough of the matching to highlight an issue, and supporting the development process by generating the QuickFix code.

What I'm trying to do is create a simple enough recipe that includes all the situations I need it, but filter it so that it doesn't get suggested in every class.

When working on recipes I try to de-risk them, in this case, I wasn't sure if I could create the private constructor so I created the QuickFix first. Then refactored the search conditions to make them more specific.

Sometimes when working on recipes I'm not sure how to perform the search, so I work on that first.

I find recipes easier to create when I build them incrementally, switching between refactoring of the QuickFix and the search.

---

You can install Sensei from within IntelliJ using "Preferences \ Plugins" (Mac) or "Settings \ Plugins" (Windows)  then just search for "sensei secure code"

The source code and recipes for this can be found in the `sensei-blog-examples` repository in the Secure Code Warrior GitHub account, in the `pojoexamples` module.

https://github.com/securecodewarrior/sensei-blog-examples

‍

Tapjacking, a combination of "tap" and "hijacking", means just that. It is an attack where the attacker hijacks the users taps and tricks him into doing something he did not intend. So how does it work and how do we prevent it?

Well start our story with screen overlays. Screen overlays or as Google calls them, windows using the type TYPE_APPLICATION_OVERLAY. These are windows that are drawn on top of other apps and usually only obscure a part of the screen. These are often used (like the example image below) when an app requests new permissions.

This is actually a cool and fun feature and more and more apps are starting to use it, think about Facebooks chat bubbles, or Google Maps navigation in a corner of your screen, like the screenshot below.

However, there are some security risks involved with these overlays. Any active screen overlay can listen for taps, how else would Facebook know that we tapped or dragged the bubble? This allows apps to spy on you and potentially steal passwords and credit card data.

One step further, and this is where the term tapjacking comes from, overlays can draw things on top of other apps tricking the user to perform different actions. The user thinks he is interacting with the overlay but in reality, his taps also perform actions in the underlying app. This way, the overlay can trick you into enabling certain permissions, or change some dangerous settings, as demonstrated in this old YouTube video.

The demonstration video above was uploaded to YouTube in 2010, so it was done on an older version of Android. But the attack is still relevant today, as vulnerabilities came to light that allows tapjacking in newer versions of Android such as Nougat and Marshmallow.

So what can you do about it? As a user, it is important to realize the consequences of these overlays and be aware of apps that use it. As of API level 23 (Android 6.0 Marshmallow), this has become a permission that has to be explicitly granted by the user. However, that leaves 50% of Androidusers still vulnerable. You can still check what apps make use of this permission in the settings under "Display over other apps".

As developers, it is up to us to make sure user actions are done with the full knowledge and consent of the user. Android provides a setting for your views that do just that, called filterTouchesWhenObscured. When it is enabled, the framework will discard touches that are received whenever the views window is obscured by another visible window. Its as simple as that, set filterTouchesWhenObscured to true, and your app is safe from tapjacking.

Good luck and see you next week!

Sometimes it is essential that an application be able to verify that an action is being performed with the full knowledge and consent of the user, such as granting a permission request, making a purchase or clicking on an advertisement. Unfortunately, a malicious application could try to spoof the user into performing these actions, unaware, by concealing the intended purpose of the view.

https://developer.android.com/reference/android/view/View.html

Device encryption is the process of encrypting all the data on the Android device. Once this is enabled for an Android device, all user-created data is immediately encrypted before it is written to the storage. This protects the data so that even if any unauthorized party tries to access the data, they wont be able to read it. This is an especially good feature for smartphone devices, since these are carried around by the user and more prone to loss or theft than other computing devices. Any curious finder or thief can not access the data unless he can unlock the phone.

Android has two types of device encryption: full-disk encryption and file-based encryption.

Full-disk encryption

Full-disk encryption was introduced in API level 19 (Android 4.4 KitKat), but the new features in API level 21 (Android 5.0 Lollipop) really kickstarted its use. Full-disk encryption uses a single key to protect the whole devices userdata partition. The key is protected with the users credentials and they must be provided upon boot before any part of the disk is accessible.

This is great for security but it also means that most of the functionality of the device is unavailable until the user enters their credentials. This means when the device is rebooted but not unlocked, some features like alarms cannot operate, services are unavailable and phones cannot receive calls. For this reason the second encryption mode was created.

File-based encryption

File-based encryption, the second mode of device encryption, has been available since API level 24 (Android 7.0 Nougat). In this mode, different files are encrypted with different keys that can be unlocked independently. With this encryption mode came the Direct Boot mode, which allows encrypted devices to boot straight to the lock screen, enabling the previously missing features before unlocking the device.

The Direct Boot allows apps to operate within a limited context before the device is unlocked. This way, they can still function as expected without compromising the user information. In order to provide this functionality, the Android device needs two storage locations:

1. Credential Encrypted storage. Default storage location, only available after the user has unlocked the device.
2. Device Encrypted storage. Available in Direct Boot mode, and after the user has unlocked the device.

If your app needs to access data while running in Direct Boot mode, it should use the Device Encrypted storage. But be aware of the security implications! Device Encrypted Storage should not be used to store any sensitive data! The Device Encrypted storage is encrypted with a key that is available as soon as the device has successfully booted. Any data that is only meant be accessed by the user should be saved in the default location, the Credential Encrypted storage.

If you want to learn more about what encryption is or why it is important, check out the video on the Secure Code Warrior portal. Or you can try to test your knowledge on encryption by playing some challenges.

Need a step-by-step guide for your device? Check out this article from Bill Hess at Pixel Privacy.

I hope you learned something new. See you next week!

Credential encrypted storage, which is the default storage location and only available after the user has unlocked the device.

https://developer.android.com/training/articles/direct-boot.html

This week we are going to talk about the default behavior of Zip libraries. If you are an application developer, it is very likely that you have used this before. Most resources that are downloaded over the internet are in zip format, this makes sense; compressed data is smaller, so it downloads faster and consumes less bandwidth.

If you want some more concrete examples: textures for games, language packs for auto-completion in keyboards, ... Many resources are not automatically bundled with the application but downloaded later.

But be cautious when using this functionality, file names in zip archives can contain path traversal information. When extracted, this will lead to files being created outside of the intended directory. This is often done with the intent of overwriting existing files.

Say we have a zip archive containing the following two files:

file1
  ../file2

When this archive is extracted, file1 is extracted where we expect it to be, in the unzip directory. However, file2 was written one directory higher than where we asked the zip library to extract the archive.

So be careful, if your zip library does not take care to properly handle this case, it will allow an attacker to write an arbitrary file in the system. Always check if your library is secure, this rule of thumb is valid for any library, but in particular you know to check the default behavior of your zip library for these types of files.

Lets demonstrate the consequences when this case is not properly handled in Android. In Android, the Java Zip library (java.util.zip) is used, the library allows path traversal as explained above by default.

Androids Dalvik Executable format (.dex) has limitations on the amount of classes a single file can have. Apps that need more classes can make use of the MultiDex Support library that has been added since API level 21 (Android 5.0 Lollipop). This library saves secondary .dex files in the data directory of the application, this directory is writable by the app user and this code will be loaded and executed when the .dex file is needed.

This means that an attacker can modify the .dex file by overwriting it using a malicious zip archive and even worse, this file will be loaded and executed, resulting in a remote code execution vulnerability. This is not merely a theoretical example but has been demonstrated on the app My Talking Tom, which has over a 100 million downloads on the app store. Here is a video of the exploit that was presented at Black Hat.

Always check the behavior of your zip library so you are aware of its insecurities. If you cannot disable path traversal in your zip library, make sure you validate the name of each entry before extracting it. The name should be canonicalized and the resulting path should be in the directory you want to extract the archive. While we are at it, you should also check the total size of the extracted archive to prevent zip bombs, but this will be a post for another week.

If you want to play some challenges on path traversal or want to test your secure coding skills, check out our platform.

See you next time, and remember, secure code or no code!

- We can inject a file into a zip whose name is prefixed with an arbitrary number of " ../ "
- If the zip library does not take care to properly handle this case, it would allow us to write outside of the intended extraction directory
- If the zip file is untrusted, this gives the attacker an arbitrary write vulnerability

https://www.blackhat.com/docs/ldn-15/materials/london-15-Welton-Abusing-Android-Apps-And-Gaining-Remote-Code-Execution.pdf