JSON deserialization to custom objects with Jackson part 1 - fields

Jackson is one of the most popular libraries (at least according to GitHub stars) used to handle a variety of tasks related to the JSON format in Java. Among other things Jackson can parse a JSON file and represent it in memory as an instance of the provided class. Of course, as long as the structure of data in the file corresponds to the structure of the class.

In all of the examples below, I’ve used the same JSON file that I’ve tried to deserialize to classes with various field modifiers and methods. All of the examples are available on my GitHub.

1
2
3
4
{
  "name": "fooName",
  "age": 23
}

No-arg constructor and field injection

Jackson uses Java reflection under the hood to construct an instance of a given class and then inject values from the JSON document to the created instance.

By default, Jackson requires a no-arg constructor to instantiate a class. Then the values are injected into fields directly (if they are public) or through a corresponding setter method. If we don’t provide any additional information to Jackson the declared field names in our class must match field names in the JSON file. The setters follow the setX convention.

Public, mutable fields

When our class has a no-arg constructor and public fields that are not final the deserialization process is straightforward and we don’t need to configure the ObjectMapper in any special way.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
package com.adebski.jackson;

import java.util.Objects;

public class PersonPublicFieldsNoArgConstructor {
    public PersonPublicFieldsNoArgConstructor() {
        System.out.println("PersonPublicFieldsNoArgConstructor constructor");
    }

    public String name;
    public int age;

     // equals, hashCode, toString, extra methods...
}

Jackson detects the name and age fields in the class, calls the no-arg constructor through reflection, and injects values directly into the fields (also through reflection).

This is verified by the following unit test.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
    @Test
    public void deserializeWithPublicFieldsNoArgConstructor() throws IOException {
        PersonPublicFieldsNoArgConstructor deserializedValue =
            objectMapper.readValue(getSamplePersonFileURL(), PersonPublicFieldsNoArgConstructor.class);

        System.out.println(deserializedValue);
        PersonPublicFieldsNoArgConstructor expectedValue = new PersonPublicFieldsNoArgConstructor();
        expectedValue.name = "fooName";
        expectedValue.age = 23;
        Assertions.assertEquals(expectedValue, deserializedValue);
    }

Private, mutable fields

Things get more interesting if we have a class with private fields that are still mutable.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
package com.adebski.jackson;

import java.util.Objects;

public class PersonPrivateFieldsNoArgConstructor {

    public PersonPrivateFieldsNoArgConstructor() {
        System.out.println("PersonPrivateFieldsNoArgConstructor constructor");
    }

    private String name;
    private int age;

    @Override
    public String toString() {
        return "PersonPrivateFieldsNoArgConstructor{" +
            "name='" + name + '\'' +
            ", age=" + age +
            '}';
    }
        
    // equals, hashCode, toString, extra methods...
}

By default, Jackson does not “see” non-public fields and throws a com.fasterxml.jackson.databind.exc.UnrecognizedPropertyException exception. The standard behavior of an ObjectMapper is to throw if a property from a JSON file does not have a corresponding property in the provided class.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    @Test
    public void deserializeWithPrivateFieldsNoArgConstructor() throws IOException {
        UnrecognizedPropertyException unrecognizedPropertyException = Assertions.assertThrows(
            UnrecognizedPropertyException.class,
            () -> objectMapper.readValue(getSamplePersonFileURL(), PersonPrivateFieldsNoArgConstructor.class)
        );

        String expectedMessagePrefix =
            "Unrecognized field \"name\" (class com.adebski.jackson.PersonPrivateFieldsNoArgConstructor), not marked as ignorable (0 known properties: ])";
        Assertions.assertTrue(
            unrecognizedPropertyException.getMessage().startsWith(expectedMessagePrefix)
        );
    }
}

On the other hand, we can configure our ObjectMapper to consider fields with visibility modifiers other than public for field injection.

1
2
3
4
5
6
7
8
9
    @Test
    public void deserializeWithPrivateFieldsNoArgConstructorWithAdditionalConfig() throws IOException {
        objectMapper.setVisibility(PropertyAccessor.FIELD, JsonAutoDetect.Visibility.ANY);

        PersonPrivateFieldsNoArgConstructor deserializedValue =
            objectMapper.readValue(getSamplePersonFileURL(), PersonPrivateFieldsNoArgConstructor.class);
        System.out.println(deserializedValue);
        Assertions.assertEquals(PersonPrivateFieldsNoArgConstructor.getExpectedValue(), deserializedValue);
    }

Here is a JavaDoc with all the possible values of com.fasterxml.jackson.annotation.JsonAutoDetect.Visibility.

Final fields

Since we already know that Jackson can inject into private fields I’ll focus only on the final fields that are also public. All things in this section apply regardless of the field visibility modifier.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
package com.adebski.jackson;

import java.util.Objects;

public class PersonPublicFieldsFinalNoArgsConstructor {

    public PersonPublicFieldsFinalNoArgsConstructor() {
        System.out.println("PersonPublicFieldsFinalNoArgsConstructor constructor");
    }

    public final String name = "initialValue";
    public final int age = -25;

    public String getName() {
        return name;
    }

    public int getAge() {
        return age;
    }

    @Override
    public String toString() {
        return "PersonPublicFieldsFinalNoArgsConstructor{" +
            "name='" + name + '\'' +
            ", age=" + age +
            '}';
    }
    
    // equals, hashCode
}

Our initial assumption may be that Jackson should not be able to modify fields declared as final in our class, but it turns out that’s false. Under the hood Jackson modifies fields through one of the set methods of the java.lang.reflect.Field class. Below is an excerpt from the set method Javadoc.

If the underlying field is final, the method throws an IllegalAccessException unless setAccessible(true) has succeeded for this Field object and the field is non-static. Setting a final field in this way is meaningful only during deserialization or reconstruction of instances of classes with blank final fields, before they are made available for access by other parts of a program. Use in any other context may have unpredictable effects, including cases in which other parts of a program continue to use the original value of this field.

What are those “unpredictable effects”? The Java Language Specification 17.5.3 expands on that point.

In some cases, such as deserialization, the system will need to change the final fields of an object after construction. final fields can be changed via reflection and other implementation-dependent means. The only pattern in which this has reasonable semantics is one in which an object is constructed and then the final fields of the object are updated. The object should not be made visible to other threads, nor should the final fields be read, until all updates to the final fields of the object are complete. Freezes of a final field occur both at the end of the constructor in which the final field is set, and immediately after each modification of a final field via reflection or other special mechanism.
Even then, there are a number of complications. If a final field is initialized to a constant expression (§15.28) in the field declaration, changes to the final field may not be observed, since uses of that final field are replaced at compile time with the value of the constant expression.

If a primitive or a String final field value is assigned directly in the class body (as opposed to writing the no-arg constructor explicitly) those values are “constant variables” by Java language (as per 4.12.4 point of the JLS. In compile-time javac replaces references to such fields with an actual value. We can observe that by using the javap utility provided by the JDK.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
javap -p -v target/classes/com/adebski/jackson/PersonPublicFieldsFinalNoArgsConstructor.class
...
// We can see that in the byte code generated for the getter a constant value is returned
  public java.lang.String getName();
    descriptor: ()Ljava/lang/String;
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: ldc           #2                  // String initialValue
         2: areturn
      LineNumberTable:
        line 15: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       3     0  this   Lcom/adebski/jackson/PersonPublicFieldsFinalNoArgsConstructor;
...
// And the toString always returns the same value, regardless of the actual value of the fields
  public java.lang.String toString();
    descriptor: ()Ljava/lang/String;
    flags: (0x0001) ACC_PUBLIC
    Code:
      stack=1, locals=1, args_size=1
         0: ldc           #9                  // String PersonPublicFieldsFinalNoArgsConstructor{name=\'initialValue\', age=-25}
         2: areturn
      LineNumberTable:
        line 24: 0
      LocalVariableTable:
        Start  Length  Slot  Name   Signature
            0       3     0  this   Lcom/adebski/jackson/PersonPublicFieldsFinalNoArgsConstructor;

If we deserialize our JSON document as PersonPublicFieldsFinalNoArgsConstructor we get some surprising results. Because getters and toString methods have the initial values hardcoded in the byte-code the instance returned by the ObjectMapper “behaves” as if it hasn’t been modified during deserialization. At the same time if we access the fields through reflection (or even in IntelliJ debugger) we will see that Jackson correctly updated the fields.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
    @Test
    public void deserializeWithPersonPublicFieldsFinalNoArgsConstructorDefaultConfiguration() throws IOException {
        PersonPublicFieldsFinalNoArgsConstructor deserializedValue =
            objectMapper.readValue(getSamplePersonFileURL(), PersonPublicFieldsFinalNoArgsConstructor.class);
        PersonPublicFieldsFinalNoArgsConstructor manuallyConstructedValue = new PersonPublicFieldsFinalNoArgsConstructor();
        System.out.println("Deserialized value: " + deserializedValue);
        System.out.println("Manually constructed value: " + manuallyConstructedValue);

        Assertions.assertEquals(
            manuallyConstructedValue,
            deserializedValue
        );
        System.out.println(
            String.format(
                "Actual name '%s' actual age '%d'",
                getNameThroughReflection(deserializedValue),
                getAgeThroughReflection(deserializedValue)
            )
        );
        Assertions.assertEquals(
            getNameThroughReflection(manuallyConstructedValue),
            manuallyConstructedValue.getName()
        );
        Assertions.assertEquals(
            getAgeThroughReflection(manuallyConstructedValue),
            manuallyConstructedValue.getAge()
        );
        Assertions.assertNotEquals(
            getNameThroughReflection(deserializedValue),
            deserializedValue.getName()
        );
        Assertions.assertNotEquals(
            getAgeThroughReflection(deserializedValue),
            deserializedValue.getAge()
        );
    }

The test prints

1
2
3
4
5
PersonPublicFieldsFinalNoArgsConstructor constructor
PersonPublicFieldsFinalNoArgsConstructor constructor
Deserialized value: PersonPublicFieldsFinalNoArgsConstructor{name='initialValue', age=-25}
Manually constructed value: PersonPublicFieldsFinalNoArgsConstructor{name='initialValue', age=-25}
Actual name 'fooName' actual age '23'

We can configure our ObjectMapper to ignore final fields during deserialization if we’d like to avoid this “surprising behavior” by setting the ALLOW_FINAL_FIELDS_AS_MUTATORS mapper feature to false. Then the deserialization results in UnrecognizedPropertyException because Jackson can’t find the fields to inject values to.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
    @Test
    public void deserializeWithPersonPublicFieldsFinalNoArgsConstructorDoNotModifyFinalFields() throws IOException {
        objectMapper.configure(MapperFeature.ALLOW_FINAL_FIELDS_AS_MUTATORS, false);

        UnrecognizedPropertyException unrecognizedPropertyException = Assertions.assertThrows(
            UnrecognizedPropertyException.class,
            () -> objectMapper.readValue(getSamplePersonFileURL(), PersonPublicFieldsFinalNoArgsConstructor.class),
            "Unrecognized field \"name\" (class com.adebski.jackson.PersonPublicFieldsFinalNoArgsConstructor), not marked as ignorable (0 known properties: ])"
        );

        String expectedMessagePrefix =
            "Unrecognized field \"name\" (class com.adebski.jackson.PersonPublicFieldsFinalNoArgsConstructor), not marked as ignorable (0 known properties: ])";
        Assertions.assertTrue(
            unrecognizedPropertyException.getMessage().startsWith(expectedMessagePrefix)
        );
    }

Conclusion

Jackson default behavior is to use a no-arg constructor and try to find a way of injecting values to fields one by one. This may or may not be the behavior we want. We’ve also seen that letting Jackson modify final fields can lead to subtle bugs which may be hard to track down and understand.

In the next post, I’ll describe the setter injection and highlight some differences between field injection and setter injection.

2021-12-25