Comparing Manifests

Manifests are Scala's solution to Java's type erasure. It is not a complete solution as of yet but it does have several useful applications. Manifests were originally added to Scala 2.7 in an extremely limited form but have been improved a great deal for Scala 2.8. Now more powerful comparisons of manifests are possible. For another introduction to Manifests (2.7 manifests) see Manifests.

This post looks at a few ways to create manifests as well as how to compare manifests. The goal is to create a method that can be used in testing to require that a certain exception is thrown during a code block:

1. scala> intercepts[Exception] {println("hi :)")}                                            
2. hi :)
3. Expected java.lang.Exception but instead no exception was raised

The code snippet above is a failure because println does not throw an exception. In addition to requiring manifests this code snippet also is a custom control structure, for more information on those see Control Structure Tag

But before writing the intercepts methods a short inspection of the new manifest comparison operators:

1. scala> import scala.reflect.{
2.      |   Manifest, ClassManifest
3.      | }
4. import scala.reflect.{Manifest, ClassManifest}
6. // from class creates a manifest object given a class object
7. scala> import ClassManifest.fromClass
8. import ClassManifest.fromClass
10. // several comparisons using <:<
11. scala> fromClass(classOf[Exception]) <:< fromClass(classOf[Exception])
12. res4: Boolean = true
14. scala> fromClass(classOf[Exception]) <:< fromClass(classOf[RuntimeException])
15. res5: Boolean = false
17. scala> fromClass(classOf[Exception]) <:< fromClass(classOf[AssertionError])         
18. res6: Boolean = false
20. // now the opposite operator >:>
21. scala> fromClass(classOf[Exception]) >:> fromClass(classOf[AssertionError])
22. res7: Boolean = false
24. scala> fromClass(classOf[Exception]) >:> fromClass(classOf[RuntimeException])
25. res8: Boolean = true
27. scala> fromClass(classOf[Exception]) >:> fromClass(classOf[Error])           
28. res9: Boolean = false
30. scala> fromClass(classOf[Exception]) >:> fromClass(classOf[Throwable])
31. res10: Boolean = false
33. // the method singleType creates a manifest given an object
34. scala> ClassManifest.singleType(new RuntimeException())
35. res12: scala.reflect.Manifest[Nothing] = java.lang.RuntimeException.type
37. scala> ClassManifest.singleType(new RuntimeException()) <:< fromClass(classOf[Throwable])
38. res13: Boolean = true
40. scala> fromClass(classOf[Exception]) <:< fromClass(classOf[Throwable])                   
41. res14: Boolean = true

Now the implementation of intercepts:

1. scala> import scala.reflect.{
2.      |   Manifest, ClassManifest
3.      | }
4. import scala.reflect.{Manifest, ClassManifest}
6. scala> 
8. scala> import ClassManifest.singleType
9. import ClassManifest.singleType
11. scala>  def intercepts[E <: Throwable](test : => Unit)(implicit m:Manifest[E]) : Unit = {
12.      |     import Predef.{println => fail}
13.      |     try {
14.      |       test
15.      |       // this is a failure because test is expected to throw an exception
16.      |       fail("Expected "+m.toString+" but instead no exception was raised")
17.      |     }catch{
18.      |       // this checks that the expected type (m) is a superclass of the class of e
19.      |       case e if (m >:> singleType(e)) => ()
20.      |       // any other error is handled here
21.      |       case e => fail("Expected "+m.toString+" but instead got "+e.getClass)
22.      |     }
23.      |   }
24. intercepts: [E <: Throwable](test: => Unit)(implicit m: scala.reflect.Manifest[E])Unit
26. scala> intercepts[Exception] {println("hi :)")}                                            
27. hi :)
28. Expected java.lang.Exception but instead no exception was raised
30. scala> intercepts[Exception] { throw new IllegalArgumentException("why not throw this :)")}
32. scala> intercepts[Exception] { throw new AssertionError("The method should complain")}     
33. Expected java.lang.Exception but instead got class java.lang.AssertionError

Matching Nulls

As a bit of explanation of one of the techniques in Regex Matching this topic reviews matching nulls.

1. // No surprise _ matches everything
2. scala> null match { case _ => println("null") }
3. null
5. // again null matches null
6. scala> null match { case null => println("null") }
7. null
9. // a is bound to anything including null
10. scala> null match { case a => println("matched value is: "+a) }
11. matched value is: null
13. scala> val a:String = null
14. a: String = null
16. // basically same as last example
17. scala> a match {case a => println( a + " is null")}          
18. null is null
20. // Any matches any non-null object
21. scala> null match {                                                
22.      | case a:Any => println("matched value is: "+a)               
23.      | case _ => println("null is not Any")
24.      | }
25. null is not Any
27. scala> val d:String = null                             
28. d: String = null
30. // In fact when matching null does not match any type
31. scala> d match {                                       
32.      | case a:String => println("matched value is: "+a)
33.      | case _ => println("no match")                   
34.      | }
35. no match
37. scala> val data:(String,String) = ("s",null)         
38. data: (String, String) = (s,null)
40. // matching can safely deal with nulls but don't forget the catch all
41. // clause or you will get a MatchError
42. scala> data match {                                  
43.      | case (a:String, b:String) => "shouldn't match"
44.      | case (a:String, _) => "should match"          
45.      | }
46. res10: java.lang.String = should match
48. // again null is all objects but will not match Any
49. scala> data match {                            
50.      | case (a:String, b:Any) => "shouldn't match"   
51.      | case (a:String, _) => "should match"       
52.      | }
53. res12: java.lang.String = should match

Regular Expression 3: Regex matching

This post covers basically the same things as Matching Regular Expressions but goes into a bit more detail. I recommend reading both posts since there is unique information in each.

The primary new item I show here is that more advanced matching techniques can be used but more importantly all groups are matched even groups that are within another group.

Note: The examples use Scala 2.8. Most examples will work with 2.7 but I believe the last example is Scala 2.8 only.

1. scala> val date = "11/01/2010"
2. date: java.lang.String = 11/01/2010
4. scala> val Date = """(\d\d)/(\d\d)/(\d\d\d\d)""".r
5. Date: scala.util.matching.Regex = (\d\d)/(\d\d)/(\d\d\d\d)
7. /*
8. When a Regex object is used in matching each group is assigned to a variable
9. */
10. scala> val Date(day, month, year) = date          
11. day: String = 11
12. month: String = 01
13. year: String = 2010
15. scala> val Date = """(\d\d)/((\d\d)/(\d\d\d\d))""".r
16. Date: scala.util.matching.Regex = (\d\d)/((\d\d)/(\d\d\d\d))
18. /*
19. This example demonstates how all groups must be assigned, if not there will be a matchError thrown
20. */
21. scala> val Date(day, monthYear, month, year) = date
22. day: String = 11
23. monthYear: String = 01/2010
24. month: String = 01
25. year: String = 2010
27. scala> val Date(day, month, year) = date           
28. scala.MatchError: 11/01/2010
29. at .< init>(< console>:5)
30. at .< clinit>(< console>)
33. // but placeholders work in Regex matching as well:
34. scala> val Date(day, _, month, year) = date
35. day: String = 11
36. month: String = 01
37. year: String = 2010
39. scala> val Names = """(\S+) (\S*)""".r             
40. Names: scala.util.matching.Regex = (\S+) (\S*)
42. scala> val Names(first, second) = "Jesse Eichar"
43. first: String = Jesse
44. second: String = Eichar
46. /*
47. If you want to use Regex's in assignment you must be sure the match will work.  Otherwise you should do real matching
48. */
49. scala> val Names(first, second) = "Jesse"       
50. scala.MatchError: Jesse
51. at .< init>(< console>:5)
52. at .< clinit>(< console>)
54. scala> val M = """\d{3}""".r
55. M: scala.util.matching.Regex = \d{3}
57. /*
58. There must be a group in the Regex or match will fail
59. */
60. scala> val M(m) = "Jan"
61. scala.MatchError: Jan
62. at .< init>(< console>:5)
63. at .< clinit>(< console>)

The following are a few more complex examples

1. scala> val Date = """((\d\d)/(\d\d)/(\d{4}))|((\w{3}) (\d\d),\s?(\d{4}))""".r         
2. Date: scala.util.matching.Regex = ((\d\d)/(\d\d)/(\d{4}))|((\w{3}) (\d\d),\s?(\d{4}))
4. /*
5. The Regex has an or in it.  So only 1/2 of the groups will be non-null.
6. If the first group is a String then it is non-null and the next three elements
7. the pattern will be day/month/year
9. Otherwise if the 5th group is a String then the patter will be month day, year
11. Lastly a catch all
12. */
13. scala> def printDate(date:String) = date match {                                      
14.      | case Date(_:String,day,month,year,_,_,_,_) => (day,month,year)                 
15.      | case Date(_,_,_,_,_:String,month,day,year) => (day,month,year) // process month
16.      | case _ => ("x","x","x")                                                        
17.      | }
18. printDate: (date: String)(String, String, String)
20. scala> printDate("Jan 01,2010") 
21. res0: (String, String, String) = (01,Jan,2010)
23. scala> printDate("01/01/2010") 
24. res1: (String, String, String) = (01,01,2010)
26. /*
27. A silly example which drops the first element of the date string
28. not useful but this demonstrates that we are matching agains a sequence so 
29. the _* can be used to match the rest of the groups
30. */
31. scala> def split(date:String) = date match {         
32.      | case d @ Date(_:String ,_*) => d drop 3       
33.      | case d @ Date(_,_,_,_,_:String,_*) => d drop 4
34.      | case _ => "boom"                              
35.      | }
36. split: (date: String)String
38. scala> split ("Jan 31,2004")
39. res5: String = 31,2004
41. scala> split ("11/12/2004") 
42. res6: String = 12/2004
44. /*
45. This is just a reminder that the findAllIn returns an iterator which (since it is probably a short iterator) can be converted to a sequence and processed with matching
46. */
47. scala> val Seq(one,two,_*) = ("""\d\d/""".r findAllIn "11/01/2010" ).toSeq  
48. one: String = 11/
49. two: String = 01/
51. scala> val Seq(one,two) = ("""\d\d/""".r findAllIn "11/01/2010" ).toSeq   
52. one: String = 11/
53. two: String = 01/
55. // drop the two first matches and assign the rest to d
56. scala> val Seq(_,_,d @ _*) = ("""\d\d/""".r findAllIn "11/01/20/10/" ).toSeq
57. d: Seq[String] = ArrayBuffer(20/, 10/)

Regular Expression 2 : The rest Regex class

This is the second installment of Regular expressions in Scala. In the first installment the basics were shown and a few of the methods in the Regex class were inspected. This topic will look at the rest of the methods in the Regex class.

Regex.findPrefixMatchOf

1. /*
2. returns the match if the regex is the prefix of the string
3. */
4. scala> "(h)(e)|(l)".r findPrefixMatchOf "hello xyz"  
5. res2: Option[scala.util.matching.Regex.Match] = Some(he)
7. scala> "lo".r findPrefixMatchOf "hello xyz"  
8. res3: Option[scala.util.matching.Regex.Match] = None
10. /*
11. The method is essentially the same as adding the boundary regex character
12. */
13. scala> "^ab".r findFirstMatchIn "ababab"
14. res8: Option[scala.util.matching.Regex.Match] = Some(ab)
16. scala> "^ab".r findFirstMatchIn "hababab"
17. res9: Option[scala.util.matching.Regex.Match] = None
19. /*
20. findPrefixOf is the same but returns the matched string instead
21. */
22. scala> "ab".r findPrefixOf "haababab"       
23. res11: Option[String] = None
25. scala> "ab".r findPrefixOf "ababab"    
26. res12: Option[String] = Some(ab)

Regex.replaceAllIn -- Essentially the same as using String.replaceAll
Regex.replaceFirstIn -- Essentially the same as using String.replaceFirst

1. scala> "(h)(e)|(l)".r replaceAllIn ("hello xyz","__")
2. res13: String = ______o xyz
4. scala> "hello xyz" replaceAll ("(h)(e)|(l)","__")    
5. res14: java.lang.String = ______o xyz
7. scala> "hello xyz" replaceFirst ("(h)(e)|(l)","__")  
8. res16: java.lang.String = __llo xyz
10. scala> "(h)(e)|(l)".r replaceFirstIn ("hello xyz","__")
11. res17: String = __llo xyz

This next section is not Scala specific but because Regex does not provide a way to set the flags CASE_INSENSITIVE, DOTALL, etc... The section is useful to demonstrate how to do it as part of the standard regex syntax.

1. // examples based on java blog at: <a href="http://www.javaranch.com/journal/2003/04/RegexTutorial.htm#flags">http://www.javaranch.com/journal/2003/04/RegexTutorial.htm#flags</a>
2. scala> val input = """Hey, diddle, diddle,      
3.      | |The cat and the fiddle,                 
4.      | |The cow jumped over the moon.           
5.      | |The little dog laughed                  
6.      | |To see such sport,                      
7.      | |And the dish ran away with the spoon.""".stripMargin
8. input: String = 
9. Hey, diddle, diddle,
10. The cat and the fiddle,
11. The cow jumped over the moon.
12. The little dog laughed
13. To see such sport,
14. And the dish ran away with the spoon.
16. // by default regex is case sensitive
17. scala> """the \w+?(?=\W)""".r findAllIn input foreach (println _)
18. the fiddle
19. the moon
20. the dish
21. the spoon
23. /* the (?i)  makes the match case insensitive the complete set of options are:
24. (?idmsux)
25. • i - case insensitive
    • d - only unix lines are recognized as end of line
    • m - enable multiline mode
    • s - . matches any characters including line end
    • u - Enables Unicode-aware case folding
    • x - Permits whitespace and comments in pattern
26. */
27. scala> """(?i)the \w+?(?=\W)""".r findAllIn input foreach (println _)
28. The cat
29. the fiddle
30. The cow
31. the moon
32. The little
33. the dish
34. the spoon

Regular Expression 1 : Basics and findAllIn

This is the first of a series on Regular expression use in Scala. There was a previous post related that is also worth looking at but the same tips will be revisited in this series: Matching Regular Expressions.

Perhaps the most important thing for regular expressions in Scala is to be aware of the raw string syntax:

1. /* 
2. normal strings treat the \ character as the escape character 
3. so this fails
4. */
5. scala> val normalString = "\.+((xyz)|(abc))"
6. < console>:1: error: invalid escape character
7.        val normalString = "\.+((xyz)|(abc))"
8.                             ^
10. /*
11. raw strings a great for regular expressions so you don't have 
12. escape \ characters
13. */
14. scala> val rawString = """\.+((xyz)|(abc))"""
15. rawString: java.lang.String = \.+((xyz)|(abc))

The next thing is to realize that one can easily create a Regex object from any string with the r method:

1. scala> val regex = """\.+((xyz)|(abc))""".r
2. regex: scala.util.matching.Regex = \.+((xyz)|(abc))

A regex has the standard matching methods one might expect but lets look at findAllIn and the associated MatchData for now. findAllIn returns a MatchIterator which is an Iterator[String] with MatchData. When iterating over the MatchIterator the full matched string will be returned, if you need the subgroups you will need to convert the MatchIterator to an Iterator[Match].

1. // findAllIn returns an iterator over the matches.  
2. scala> "l|he".r findAllIn "hello xyz" foreach {println _}
3. he
4. l
5. l
7. // Each match can have multiple groups
8. // Note: Each element in MatchIterator are strings (no Match objects)
9. scala> ("(h)(e)|(l)".r findAllIn "hello xyz").matchData foreach { m => println(m.matched mkString ",")}   
10. h,e
11. l
12. l
14. // to access subgroups use the matchData method
15. // Note: there are 3 subgroups in the regex
16. scala> ("(h)(e)|(l)".r findAllIn "hello xyz").matchData foreach { m => println( m.subgroups mkString ",")}
17. h,e,null
18. null,null,l
19. null,null,l
21. /*
22. if matched is called the full match is returned (as if you did not convert the iterator to an Iterator[Match])
23. /*
24. scala> ("(h)(e)|(l)".r findAllIn "hello xyz").matchData foreach { m => println(m.matched)}
25. he
26. l
27. l
29. /*
30. The following demonstrates more of the methods on Match
31. Essentially the elements are:
32. (start index of match, end index of match, string before match, string after match, string the match was performed on)
33. */
34. scala> ("(h)(e)|(l)".r findAllIn "hello xyz").matchData foreach { m => println(m.start, m.end, m.before, m.after, m.source)}
35. (0,2,,llo xyz,hello xyz)
36. (2,3,he,lo xyz,hello xyz)
37. (3,4,hel,o xyz,hello xyz)

The last methods to look at for this topic are findFirstIn, findFirstMatchIn and the Regex constructor:

1. /*
2. Groups names can be assigned if the Regex constructor is used
3. */
4. scala> val withNames = new util.matching.Regex("(h)(e)|(l)", "h", "e", "l")
5. withNames: scala.util.matching.Regex = (h)(e)|(l)
7. scala> withNames findFirstIn "hello xyz"
8. res28: Option[String] = Some(he)
10. /*
11. I know a match will be found so I am extracting the value from the Option by assigning it to Some(he)
12. */
13. scala> val Some(he) = withNames findFirstMatchIn "hello xyz"  
14. he: scala.util.matching.Regex.Match = he
16. scala> he.groupNames
17. res29: Seq[String] = Array(h, e, l)
19. scala> he.group("h")
20. res30: String = h
22. scala> he.group("e") 
23. res31: String = e
25. scala> he.group(1)
26. res32: String = h
28. scala> he.group(2)
29. res33: String = e
31. // Uh oh. NullPointer warning!
32. scala> he.group(3)
33. res34: String = null
35. scala> he.groupCount
36. res35: Int = 3

Matching in for-comprehensions

At a glance a for-comprehension appears to be equivalent to a Java for-loop, but it is much much more than that. As shown in post: for-comprehensions, for-comprehensions can have guards which filter out which elements are processed:

1. scala> for ( x <- 1 to 10; if (x >4) ) println(x)
2. 5
3. 6
4. 7
5. 8
6. 9
7. 10

They can be used to construct new collections:

1. scala>?for(?i?<-?List(?"a",?"b",?"c")?)?yield?"Word:?"+i
2. res1:?List[java.lang.String]?=?List(Word:?a,?Word:?b,?Word:?c)

They can contain multiple generators:

1. scala> for {x <- 1 to 10                           
2.      |      if(x%2 == 0)
3.      |      y <- 1 to 5} yield (x,y)
4. res1: scala.collection.immutable.IndexedSeq[(Int, Int)] = IndexedSeq((2,1), (2,2), (2,3), (2,4), (2,5), (4,1), (4,2), (4,3), (4,4), (4,5), (6,1), (6,2), (6,3), (6,4), (6,5), (8,1), (8,2), (8,3), (8,4), (8,5), (10,1), (10,2), (10,3), (10,4), (10,5))

What has not been covered is that the assignments also does pattern matching:

1. scala> for ( (x,y) <- (6 to 1 by -2).zipWithIndex) println (x,y) 
2. (6,0)
3. (4,1)
4. (2,2)

This is not surprising as this also occurs during normal assignment. But what is interesting is that the pattern matching can act as a guard as well. See Extractor examples and Assignment and Parameter Objects for more information of pattern matching and extractors.

1. scala> val args = Array( "h=2", "b=3")
2. args: Array[java.lang.String] = Array(h=2, b=3)
4. scala> val Property = """(.+)=(.+)""".r 
5. Property: scala.util.matching.Regex = (.+)=(.+)
7. scala> for {Property(key,value) <- args } yield (key,value)
8. res0: Array[(String, String)] = Array((h,2), (b,3))
10. scala> Map(res0:_*)
11. res1: scala.collection.immutable.Map[String,String] = Map(h -> 2, b -> 3)
13. scala> res1("h")
14. res3: String = 2

Now just for fun here is a similar example but using symbols instead of strings for the key values:

1. scala> val args = Array( "h=2", "b=3")
2. args: Array[java.lang.String] = Array(h=2, b=3)
4. scala> val Property = """(.+)=(.+)""".r 
5. Property: scala.util.matching.Regex = (.+)=(.+)
7. scala> for {Property(key,value) <- args } yield (Symbol(key),value)
8. res0: Array[(Symbol, String)] = Array(('h,2), ('b,3))
10. scala> Map(res0:_*)
11. res1: scala.collection.immutable.Map[Symbol,String] = Map('h -> 2, 'b -> 3)
13. scala> res1('h)
14. res2: String = 2

Symbols

Scala has what are called symbolic literals. A symbol is very similar to a String except that they are cached. So symbol 'hi will be the same object as 'hi declared a second time. In addition there is special syntax for creating symbols that requires only a single quote.

As odd as this may appear at first glance there are some good use cases for symbols.

• Save memory
• Extra semantics. Semantically they are the same object where "hi" is not necessarily the same as "hi" so 'hi has some extra semantics meaning this is the one an only 'hi object.
• Shorter syntax. This can be useful when designing DSLs
• Identifier syntax. When using the simpler syntax the symbol will be a valid Scala identifier so it can be good when intending to reference methods or variable, perhaps in a heavily reflection based framework

I hope that makes sense :)

1. // This is the long way to create a symbol object
2. scala> Symbol("hi")
3. res10: Symbol = 'hi
5. // This is the short way.  The symbol must be a legal Scala symbol (like a method name or value/variable name)
6. scala> 'hi
7. res11: Symbol = 'hi
9. // If you *need* characters in a symbol that are not legal in Scala identifiers the Symbol 
10. // object has a factory method for that purpose
11. scala> Symbol("hi there")
12. res12: Symbol = 'hi there
14. // Not legal
15. scala> 'hi there
16. < console>:5: error: value there is not a member of Symbol
17.        'hi there
18.            ^
20. // quotes are not legal for identifiers
21. scala> '"hi there"
22. < console>:1: error: unclosed character literal
23.        '"hi there"
24.        ^
25. < console>:1: error: unclosed string literal
26.        '"hi there"
27.                  ^
29. scala> 'hi\ there 
30. < console>:5: error: value \ is not a member of Symbol
31.        'hi\ there
32.        ^
34. // You can extract out the string from the symbol if desired quite easily
35. scala> 'hi match { case Symbol(b) => b}        
36. res14: String = hi
38. // A simpler way to get the string
39. scala> 'hi.toString drop 1
40. res0: String = hi