NullPointer when mixed traits (Warning)

This tip is mainly to document a 'GOTCHA' that I got caught by recently. It basically goes like this:

Trait Y extends(or has self-type) X. Trait X defines some abstract method 'm'. The initialization code in Y accesses 'm'. Creation of an object new X with Y results in: *Boom* NullPointerException (on object creation).

The example in code:

1. scala> trait X { val x : java.io.File }
2. defined trait X
4. scala> trait Y {self : X => ; val y = x.getName} 
5. defined trait Y
7. scala> new X with Y { val x = new java.io.File("hi")}
8. java.lang.NullPointerException
9. at Y$class.$init$(< console>:5)
10. at $anon$1.< init>(< console>:7)
11. ...

At a glance it seems that x should override the abstract value x in trait X. However the order in which traits are declared is important. In this case first Y is configured then X. Since X is not yet configured Y throws an exception. There are several ways to work around this.
Option 1:

1. trait X {val x : java.io.File}
2. trait Y {self : X => ; val y = x.getName}
3. /*
4. Declaring Y with X will work because Y is initialized after X
5. but remember that there may
6. be other reasons that X with Y is required.  
8. Method resolution is one such reason
9. */
10. new Y with X { val x = new java.io.File("hi")}

Option 2:

1. trait X { val x : java.io.File }
2. trait Y {self : X => ; def y = x.getName}
3. /*
4. Since method y is a 'def' x.getName will not be executed during initialization.
5. */
6. scala> new X with Y { val x = new java.io.File("hi")}
7. res10: java.lang.Object with X with Y = $anon$1@7cb9e9a3

Option 3:

1. trait X { val x : java.io.File }
2. trait Y {self : X => ; lazy val y = x.getName}
3. /*
4. 'lazy val' works for the same reason 'def' works: x.getName is not invoked during initialization
5. */
6. scala> new X with Y { val x = new java.io.File("hi")}
7. res10: java.lang.Object with X with Y = $anon$1@7cb9e9a3

Option 4:

1. trait X {val x : java.io.File }
2. trait Y extends X {def y = x.getName}
3. /*
4. if Y extends X then a new Y can be instantiated
5. */
6. new Y {val x = new java.io.File("hi")}

Two more warnings. First, the same error will occur whether 'x' is a def or a val or a var.

1. trait X { def x : java.io.File }   
2. trait Y {self : X => ; val y = x.getName}     
3. new X with Y { val x = new java.io.File("hi")}

Second warning: In complex domain models it is easy to have a case where Y extends X but the final object is created as: new X with Y{...}.

You will get the same error here because (I think) the compiler recognized that Y is being mixed in with X and therefore the X will be initialized as after Y instead of before Y.

First the code:

1. trait X { def x : java.io.File }   
2. trait Y extends X { val y = x.getName}        
3. new X with Y { val x = new java.io.File("hi")}

If the code instantiated new Y{...} the initialization would be X then Y. Because X can only be initialized once, the explicit declaration of new X with Y forces Y to be initialized before X. (X can only be initialized once even when it appears twice in the hierarchy).

This is a topic called linearization and will be addressed in the future.

Self Annotation vs inheritance

At first glance the "sself annotation" declaration seems similar to extending another class. (For a look at self annotations read the topic: Self Type.) They are completely different but the comparison is understandable since both of them provide access to the functionality of referenced class.

For example both of the following compile:

1. class Base {
2.   def magic = "bibbity bobbity boo!!"
3. }
5. trait Extender extends Base {
6.   def myMethod = "I can "+magic
7. }
9. trait SelfTyper {
10.   self : Base => 
11.   
12.   def myMethod = "I can "+magic
13. }

But the two are completely different. Extender can be mixed in with any class and adds both the "magic" and "myMethod" to the class it is mixed with. SelfType can only be mixed in with a class that extends Base and SelfTyper only adds the method "myMethod" NOT "magic".

Why is the "self annotations" useful? Because it allows several provides a way of declaring dependencies. One can think of the self annotation declaration as the phrase "I am useable with" or "I require a".

The following example demonstrates one possible reason to use self annotations instead of extend.

Note: These examples can be pasted into the REPL but I have shown that here because it would make the examples too long.

1. import java.io._
2. import java.util.{Properties => JProps}
4. trait Properties {
5.   def apply(key:String) : String
6. }
8. trait XmlProperties extends Properties {
9.   import scala.xml._
10.   
11.   def xml(key:String) = Elem(null,key,Null,TopScope, Text(apply(key)))
12. }
14. trait JSonProperties extends Properties {
15.   def json(key:String) : String = "%s : %s".format(key, apply(key))
16. }
18. trait StreamProperties {
19.   self : Properties => 
20.   
21.   protected def source : InputStream
22.   
23.   private val props = new JProps()
24.   props.load(source)
25.   
26.   def apply(key:String) = props.get(key).asInstanceOf[String]
27. }
29. trait MapProperties {
30.   self : Properties =>
31.   
32.   protected def source : Map[String,String]
33.   def apply(key:String) = source.apply(key)
34. }
36. val sampleMap = Map("1" -> "one", "2" -> "two", "3" -> "three")
37. val sampleData = """1=one
38.                     2=two
39.                     3=three"""
41. val sXml = new XmlProperties() with StreamProperties{
42.               def source = new ByteArrayInputStream(sampleData.getBytes)
43.            }
44. val mXml = new XmlProperties() with MapProperties{
45.               def source = sampleMap
46.            }
48. val sJSon = new JSonProperties() with StreamProperties{
49.               def source = new ByteArrayInputStream(sampleData.getBytes)
50.             }
51. val mJSon = new JSonProperties() with MapProperties{
52.               def source = sampleMap
53.             }
55. sXml.xml("1")
56. mXml.xml("2")
57. sJSon.json("1")
58. mJSon.json("2")

The justification for using self annotations here is flexibility. A couple other solutions would be

1. Use subclassing - this is poor solution because there would be an explosion of classes. Instead of having 5 traits you would need 7 traits. Properties, XmlProperties, JSonProperties, XmlStreamProperties, XmlMapProperties, JsonStreamProperties and JsonMapProperties. And if you later wanted to add a new type of properties or a new source like reading from a database then you need 2 new subclasses.
2. Composition - Another strategy is to use construct the XmlProperties with a strategy that reads from the source. This is essentially the same mechanism except that you need to build and maintain the the dependencies. It also makes layering more difficult. For example:
   
   1. trait IterableXmlProperties {                                            
   2.     self : MapProperties with XmlProperties => 
   3.     def xmlIterable = source.keySet map {xml _}
   4.   }
   5.   
   6.   new XmlProperties with MapProperties with IterableXmlProperties {def source = sampleMap}

The next question that comes to mind is why use extends then if self annotation is so flexible? My answer (and I welcome discussion on this point) has three points.

1. The first is of semantics and modeling. When designing a model it is often more logical to use inheritance because of the semantics that comes with inheriting from another object.
2. Another argument is pragmatism.
   Imagine the collections library where there is no inheritance. If you wanted a map with Iterable functionality you would have to always declare Traversable with Iterable with Map (and this is greatly simplified). That declaration would have to be used for virtually all methods that require both the Iterable and Map functionality. To say that is impractical is an understatement. Also the semantics of Map is changed from what it is now. The trait Map currently includes the concept of Iterable.
3. The last point is sealed traits/classes. When a trait is "sealed" all of its subclasses are declared within the same file and that makes the set of subclasses finite which allows certain compiler checks. This (as far as I know) cannot be done with self annotations.

Traits and inheritance

Scala provides two structures for inheritance. Classes (abstract or not) and traits. Traits are very similar to Ruby Mixins meaning that they can contain code like abstract classes but like interfaces multiple traits can be inherited from.
Like interfaces traits cannot have constructors but in Scala variables can be abstract and therefore provide an easy way to simulate a constructor.
There are no method resolution conflicts because method definitions are always resolved right to left:

1. class X extends Y with A with B with C

If ABC and Y all have the method (doit) the method in C will be used. If C calls super.doit that will call B.doit and so on.
Note: If Y defines doit the A, B, and C must define doit with the override keyword:

1. override def doit() = {...}

In the above example ABC must be traits but Y can be a class or a trait. When inheriting you must always have one extends keyword which can optionally followed by one or more with clauses.

1. scala> abstract class Animal {
2.      |  val legs:Int
3.      | val noise:String
4.      | def makeNoise() = println(noise)
5.      | }
6. defined class Animal
8. scala>  trait Quadriped {
9.      | self:Animal =>
10.      | val legs = 4
11.      | }
12. defined trait Quadriped
14. scala> trait Biped {
15.      | self:Animal =>
16.      | val legs = 2
17.      | }
18. defined trait Biped
20. scala> class Dog extends Animal with Quadriped {
21.      | val noise = "Woof"
22.      | override def makeNoise() = println( noise+" "+noise)
23.      | }
24. defined class Dog
26. scala> new Dog().makeNoise()
27. Woof Woof
29. scala> abstract class GenericAnimal extends Animal{ 
30.      | val noise = "glup"                          
31.      | }
32. defined class GenericAnimal
34. scala> val quad = new GenericAnimal() with Quadriped
35. quad: GenericAnimal with Quadriped = $anon$1@10bfb545
37. scala> quad.makeNoise()
38. glup
40. scala> val biped = new GenericAnimal() with Biped
41. biped: GenericAnimal with Biped = $anon$1@7669521
43. scala> val biped = new GenericAnimal() with Biped{
44.      | override val noise = "Hello"
45.      | }
46. biped: GenericAnimal with Biped = $anon$1@6366ce5f
48. scala> biped.makeNoise()
49. Hello