Xml Transformation 1

Unlike most Java Xml Apis the Scala Object model consists of immutable object. This has two major consequences:

• There is no reference to the parent node because that would cause the XML to be very expensive during transformations
• Transforming the XML requires creating new nodes rather than changing the existing nodes

Both point cause non-functional programmers to feel a little uneasy but in practice only the first restriction causes any real discomfort.

Two methods for XML transformation will be demonstrated in this and the next topic.

1. scala> val xml = <library>
2.      | <videos>
3.      | <video type="dvd">Seven</video>
4.      | <video type="blue-ray">The fifth element</video>
5.      | <video type="hardcover">Gardens of the moon</video>
6.      | </videos>
7.      | <books>
8.      | <book type="softcover">Memories of Ice</book>
9.      | </books>
10.      | </library>
11. xml: scala.xml.Elem = 
12. <library>
13.        <videos>
14.        <video type="dvd">Seven</video>
15.        <video type="blue-ray">The fifth element</video>
16.        <video type="hardcover">Gardens of the moon</video>
17.        </videos>
18.        <books>
19.        <book type="softcover">Memories of Ice</book>
20.        </books>
21.        </library>
23. scala> import scala.xml._
24. import scala.xml._
26. scala> import scala.xml.transform._
27. import scala.xml.transform._
29. // Some of the books are labelled as videos
30. // not books so lets select those elements
31. scala> val mislabelledBooks = xml \\ "video" filter {e => (e \\ "@type").text == "hardcover"}
32. mislabelledBooks: scala.xml.NodeSeq = <video type="hardcover">Gardens of the moon</video>
34. // we can create a rule that will remove all the
35. // selected elements
36. scala> object RemoveMislabelledBooks extends RewriteRule {
37.      | override def transform(n: Node): Seq[Node] ={ 
38.      | if (mislabelledBooks contains n) Array[Node]()
39.      | else n
40.      | }
41.      | }
42. defined module RemoveMislabelledBooks
44. // a quick test to make sure the elements are removed
45. scala> new RuleTransformer(RemoveMislabelledBooks)(xml)
46. res1: scala.xml.Node = 
47. <library>
48.        <videos>
49.        <video type="dvd">Seven</video>
50.        <video type="blue-ray">The fifth element</video>
51.        
52.        </videos>
53.        <books>
54.        <book type="softcover">Memories of Ice</book>
55.        </books>
56.        </library>
59. // Now another rule to add them back
60. scala> object AddToBooks extends RewriteRule {                             
61.      | override def transform(n: Node): Seq[Node] = n match {                                
62.      | case e:Elem if(e.label == "books") =>                                                 
63.      |   val newBooks = mislabelledBooks map { case e:Elem => e.copy(label="book") }
64.      |   e.copy(child = e.child ++ newBooks)                                                 
65.      | case _ => n
66.      | }
67.      | }
68. defined module AddToBooks
70. // voila done
71. scala> new RuleTransformer(RemoveMislabelledBooks, AddToBooks)(xml) 
72. res4: scala.xml.Node = 
73. <library>
74.        <videos>
75.        <video type="dvd">Seven</video>
76.        <video type="blue-ray">The fifth element</video>
78.        </videos>
79.        <books>
80.        <book type="softcover">Memories of Ice</book>
81.        <book type="hardcover">Gardens of the moon</book></books>
82.        </library>

Futures

Futures are mainly tokens returned by an actor that promises a result sometime in the future. The typical place one encounters a future is from an actor. The !! method of an actor returns a future. However in addition the Futures object provides several methods for doing parallel processing without having to write actors (or threads for that matter.)

Note: My use of scala.util.Random requires Scala 2.8 but I think the rest is 2.7.7.

1. scala> import scala.actors.Futures._                
2. import scala.actors.Futures._
4. /* 
5. The most basic future example.  
6. Just runs the function in a seperate thread and returns a Future object
7. for the result
8. */
9. scala> future {Thread.sleep(10000);println("hi");10}
10. res2: scala.actors.Future[Int] = < function0>
12. /* 
13. Use one of the future methods for obtaining the actual result
14. this one blocks tile ready, but other can check if result is ready and so on
15. */
16. scala> res2()
17. hi
18. res3: Int = 10
20. /*
21. This is more interesting. 
22. The method creates several futures and the using the awaitAll method 
23. waits for all the futures to complete before continuing
24. */
25. scala> def bubbles = {                                                                                                             
26.      | val bubbles = for( i <- 1 to 20) yield {
27.      |   future {
28.      |     Thread.sleep(scala.util.Random.nextInt(20)*500)
29.      |     println("pop "+i)
30.      |     "pop "+i
31.      |   }
32.      | }
33.      | awaitAll(30000, bubbles:_*) foreach println _                                                                               
34.      | }
35. bubbles: Unit
37. scala> bubbles
38. pop 9
39. pop 12
40. pop 11
41. pop 5
42. pop 3
43. pop 14
44. pop 20
45. pop 10
46. pop 1
47. pop 4
48. pop 16
49. pop 6
50. pop 7
51. pop 2
52. pop 8
53. pop 18
54. pop 15
55. pop 19
56. pop 17
57. pop 13
58. Some(pop 1)
59. Some(pop 2)
60. Some(pop 3)
61. Some(pop 4)
62. Some(pop 5)
63. Some(pop 6)
64. Some(pop 7)
65. Some(pop 8)
66. Some(pop 9)
67. Some(pop 10)
68. Some(pop 11)
69. Some(pop 12)
70. Some(pop 13)
71. Some(pop 14)
72. Some(pop 15)
73. Some(pop 16)
74. Some(pop 17)
75. Some(pop 18)
76. Some(pop 19)
77. Some(pop 20)
79. scala> import java.net._                                                                                        
80. import java.net._
82. scala> import scala.io._
83. import scala.io._
85. /*
86. A kind of funny example.  We set off two futures to download the google page
87. and print the first that finishes
88. */
89. scala> def load = {
90.      | val com = future {
91.      |   val comStream = new URL("http://www.google.com").openStream()                                                                       
92.      |   Source.fromInputStream(comStream).getLines().mkString("\n")
93.      | }
94.      | val ch = future {
95.      |   val chStream = new URL("http://www.google.ch").openStream()
96.      |   Source.fromInputStream(chStream).getLines().mkString("\n")
97.      | }  
98.      | awaitEither(ch,com)                                                                                               
99.      | }
100. load: Any
102. scala> load
103. res0: Any = 
104. & !doctype html...

Labelled Multiple Returns

A cool idea I found at http://stackoverflow.com/questions/1827672/is-there-a-way-to-have-tuples-with-named-fields-in-scala-similar-to-anonymous-cl. A very simple idea but very useful.
To summarize, you want to return multiple values but want to assign names to the values rather than use a tuple which more or less assigns indices to the different values. Here are a couple of solutions:

Using structured types. Simplest solution with the least extra code.

1. scala> def multipleReturns1 = {
2.      | new { val first = 1; val second = 2 }
3.      | }
4. multipleReturns1: java.lang.Object{def first: Int; def second: Int}
6. scala> multipleReturns1
7. res0: java.lang.Object{def first: Int; def second: Int} = $anon$1@549b6976
9. scala> res0.first
10. res1: Int = 1
12. scala> res0.second
13. res2: Int = 2
15. scala> import res0._
16. import res0._
18. scala> first
19. res3: Int = 1
21. scala> second
22. res4: Int = 2

Using case classes. Returns a product which has properties that may be useful.

1. scala> def moreReturns = {
2.      | case class Returns (one:Int, two:Int)
3.      | Returns(1,2)
4.      | }
5. moreReturns: java.lang.Object with ScalaObject with Product{def one: Int; def two: Int}
7. scala> moreReturns
8. res5: java.lang.Object with ScalaObject with Product{def one: Int; def two: Int} = Returns(1,2)
10. scala> res5.one
11. res6: Int = 1
13. scala> res5.two
14. res7: Int = 2
16. scala> import res5._
17. import res5._
19. scala> one
20. res8: Int = 1
22. scala> two
23. res9: Int = 2
25. scala> res5.productIterator foreach println _
26. 1
27. 2
29. scala> res5.productPrefix
30. res15: java.lang.String = Returns

Scala cheat sheet

One can never go wrong with a Cheat sheet. It is not all there but it is a nice start and well worth looking through.

http://anyall.org/scalacheat/

Scala 2.8 Traversable and IndexedSequence

This topic is really just a collection of examples using the Scala 2.8 API. The examples are mostly based on Strings. But because Strings are in fact an IndexedSeq all of these examples also work with Lists arrays and the first several work with all collections.

These examples are all based on first Traversable. The basis of all collections and therefore will work with all collections. And the next set of examples work with IndexedSeq and most will work with Seq as well. Sets, Maps and others still need to be examined.

This is not an exhaustive set of examples and not all are even that interesting (sorry :-) ). But I let out the most trivial unless theay are being compared to others.

Traversable:

1. scala> val license = """Subject of Agreement. ?Product?, as referred to in this Agreement, shall be the binary software package ?Sun VirtualBox,? which Product allows for creating multiple virtual computers, each with different operating systems (?Guest Computers?), on a physical computer with a specific operating system (?Host Computer?), to allow for installing and executing these Guest Computers simultaneously. The Product consists of executable files in machine code for the Solaris, Windows, Linux, and Mac?OS?X operating systems as well as other data files as required by the executable files at run-time and documentation in electronic form. The Product includes all documentation and updates provided to You by Sun under this Agreement and the terms of this Agreement will apply to all such documentation and updates unless a different license is provided with an update or documentation."""
2. res0: RandomAccessSeq[Char] = 
3. license: java.lang.String = ...
5. scala> license count (_ == 'u')
6. res1: Int = 37
8. scala> license split ' ' count {_ == "and"} 
9. res3: Int = 6
10. scala> license ++ " this is silly!"
11. res0: RandomAccessSeq[Char] = ...
13. scala> license dropWhile { _ != 'y'} drop 1 takeWhile { _ != 'y'}
14. res6: Seq[Char] = RandomAccessSeqTW( , s, o, ...
16. scala> license forall {_ != 'z'}
17. res8: Boolean = true
19. scala> (1 until 5).init
20. res0: scala.collection.immutable.Range = Range(1, 2, 3)
22. scala> (1 until 5).head
23. res1: Int = 1
25. scala> (1 until 5).tail
26. res2: scala.collection.immutable.IndexedSeq[Int] = Range(2, 3, 4)
28. scala> (1 until 5).last
29. res3: Int = 4
31. scala> (1 until 5).headOption
32. res4: Option[Int] = Some(1)
34. scala> (1 until 5).lastOption
35. res6: Option[Int] = Some(4)
37. scala> license max   
38. res8: Char = y
40. scala> license min
41. res9: Char =  
43. scala> List() nonEmpty 
44. res10: Boolean = false
46. scala> List() isEmpty 
47. res11: Boolean = true
49. scala> license partialMap {case c if ('a' to 'z' contains c) => c}
50. res13: scala.collection.immutable.IndexedSeq[Any] = IndexedSeq(u, b, j, e, c ...)
52. scala> 1 to 10000 by 2 product 
53. res19: Int = 481029393
55. scala> 1 to 10000 by 2 sum                   
56. res23: Int = 25000000
58. scala> license partition {"aeiou" contains _}
59. res20: (String, String) = (ueoeeeouaeeeoiieeeaeeia ....
61. scala> license span {_ != 'a'}
62. res22: (String, String) = (Subject of Agreement. ?Product?, ,as referred to in this Agreement, shall be the binary software package ?Sun VirtualBox,? which Product allows for creating multiple virtual computers, each with differ ...
64. scala> val consonants = license withFilter {c => !("aeiuo" contains c)}
65. consonants: scala.collection.immutable.StringOps#WithFilter = scala.collection.TraversableLike$WithFilter@78e8a591
67. scala> consonants foreach print _                                      
68. Sbjct f Agrmnt. ?Prdct?, s rfrrd t n ths Agrmnt, shll b th bnry sftwr pckg ?Sn VrtlBx,? whch Prdct llws fr crtng mltpl vrtl cmptrs, ch wth dffrnt prtng systms (?Gst Cmptrs?), n  physcl cmptr wth  spcfc prtng systm (?Hst Cmptr?), t llw fr nstllng nd xctng ths Gst Cmptrs smltnsly. Th Prdct cnssts f xctbl fls n mchn cd fr th Slrs, Wndws, Lnx, nd Mc?OS?X prtng systms s wll s thr dt fls s rqrd by th xctbl fls t rn-tm nd dcmnttn n lctrnc frm. Th Prdct nclds ll dcmnttn nd pdts prvdd t Y by Sn ndr ths Agrmnt nd th trms f ths Agrmnt wll pply t ll sch dcmnttn nd pdts nlss  dffrnt lcns s prvdd wth n pdt r dcmnttn.

IndexedSequense additions to Traversable:

1. scala> '!' +: license    
2. res33: String = !Subject of Agreement
4. scala> "hello" +: license
5. res32: scala.collection.immutable.IndexedSeq[Any] = IndexedSeq(hello, S, u, b, j, e, c, t,  , o, f,  , A, g, r, e, e, m, e, n, t, .,  , ?, P, r, o, d, u, c, t, ?, ,,  , a, s,  , r, e, f, e, r, r, e, d,  , t, o,  , i, n,  , t, h, i, s,  , A, g, r, e, e, m, e, n, t, ,,  , s, h, a, l, l,  , b, e,  , t, h, e,  , b, i, n, a, r, y,  , s, o, f, t, w, a, r, e,  , p, a, c, k, a, g, e,  , ?, S, u, n,  , V, i, r, t, u, a, l, B, o, x, ,, ?,  , w, h, i, c, h,  , P, r, o, d, u, c, t,  , a, l, l
7. scala> license(11)
8. res35: Char = A
10. scala> license diff ("aeiou")
11. res47: String = Sbjct f Agreement. ?Product?, s referred to n this Agreement, shall be the binary software package ?Sun VirtualBox,? which Product allows for creating multiple virtual computers, each with different operating systems (?Guest Computers?), on a physical computer with a specific operating system (?Host Computer?), to allow for installing and executing these Guest Computers simultaneously. The Product consists of executable files in machine code for the Solaris, Windows, Linux, and Mac?OS?X operating systems as well as other data files as required by the executable files at run-time and documentation in electronic form. The Product includes all documentation and updates provided to You by Sun under this Agreement and the terms of this Agreement will apply to all such documentation and updates unless a different license is provided with an update or documentation.
13. scala> license dropRight(30)
14. res48: String = Subject of Agreement. ?Product?, as referred to in this Agreement, shall be the binary software package ?Sun VirtualBox,? which Product allows for creating multiple virtual computers, each with different operating systems (?Guest Computers?), on a physical computer with a specific operating system (?Host Computer?), to allow for installing and executing these Guest Computers simultaneously. The Product consists of executable files in machine code for the Solaris, Windows, Linux, and Mac?OS?X operating systems as well as other data files as required by the executable files at run-time and documentation in electronic form. The Product includes all documentation and updates provided to You by Sun under this Agreement and the terms of this Agreement will apply to all such documentation and updates unless a different license is provided wi
16. scala> license takeRight(10)
17. res49: String = mentation.
19. scala> license indexWhere { "aeiou" contains _} 
20. res4: Int = 1
22. scala> license lastIndexWhere { "aeiou" contains _}
23. res6: Int = 873
25. scala> List(1,2,3) flatten {0 to _}
26. res7: List[Int] = List(0, 1, 0, 1, 2, 0, 1, 2, 3)
28. scala> ' ' +: license zip license indexWhere {case (last, c) => "" + last + c == "an"}       
29. res14: Int = 343
31. scala> license indexOfSlice ("and")
32. res17: Int = 342
34. scala> ('a' to 'z') ++ ('A' to 'Z') 
35. res25: scala.collection.IndexedSeqView[Char,IndexedSeq[_]] = IndexedSeqViewA(a, b, c, d, e, f, g, h, i, j, k, l, m, n, o, p, q, r, s, t, u, v, w, x, y, z, A, B, C, D, E, F, G, H, I, J, K, L, M, N, O, P, Q, R, S, T, U, V, W, X, Y, Z)
37. scala> license intersect res25
38. res26: String = SubjectofAgrmnPdasihlywpkVBxvGCHTWLMOXqY
40. scala> license diff res25     
41. res27: String =   eeet. ?rouct?,  referred to n tis Agreement, shal be the binar softare acage ?Sun irtualo,? which Product allows for creating multiple irtual computers, each with different operating systems (?uest omputers?), on a physical computer with a specific operating system (?ost Computer?), to allow for installing and executing these Guest Computers simultaneously. he Product consists of executable files in machine code for the Solaris, indows, inux, and ac?S? operating systems as well as other data files as reuired by the executable files at run-time and documentation in electronic form. The Product includes all documentation and updates provided to ou by Sun under this Agreement and the terms of this Agreement will apply to all such documentation and updates unless a different license is provided with an update or documentation.
43. scala> license filter res25
44. < console>:7: error: type mismatch;
45.  found   : scala.collection.IndexedSeqView[Char,IndexedSeq[_]]
46.  required: (Char) => Boolean
47.        license filter res25
48.                       ^
50. scala> license filter {res25 contains _}
51. res29: String = SubjectofAgreementProductasreferredtointhisAgreementshallbethebinarysoftwarepackageSunVirtualBoxwhichProductallowsforcreatingmultiplevirtualcomputerseachwithdifferentoperatingsystemsGuestComputersonaphysicalcomputerwithaspecificoperatingsystemHostComputertoallowforinstallingandexecutingtheseGuestComputerssimultaneouslyTheProductconsistsofexecutablefilesinmachinecodefortheSolarisWindowsLinuxandMacOSXoperatingsystemsaswellasotherdatafilesasrequiredbytheexecutablefilesatruntimeanddocumentationinelectronicformTheProductincludesalldocumentationandupdatesprovidedtoYoubySununderthisAgreementandthetermsofthisAgreementwillapplytoallsuchdocumentationandupdatesunlessadifferentlicenseisprovidedwithanupdateordocumentation
53. scala> license filterNot {res25 contains _}
54. res30: String =   . ??,      ,       ? ,?        ,      (? ?),          (? ?),          .            , , ,  ??                -     .                                        .
56. scala> license removeDuplicates
57. res31: String = Subject ofAgrmn.?Pd,asihlywpkVBxv(GC)HTWLMOXq-Y
59. scala> license segmentLength ({_ > 'A'},2) 
60. res37: Int = 5
62. scala> license drop 2 span {_ > 'A'} _1
63. res41: String = bject
65. scala> license sortWith Ordering.Char          
66. res45: String =                                                                                                                                   (()),,,,,,,,,-....??????????AAAABCCCGGHLMOPPPPSSSSSTTVWXYaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabbbbbbbccccccccccccccccccccccccccccccddddddddddddddddddddddddddddddeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeefffffffffffffffffggggggggggghhhhhhhhhhhhhhhhhhhhhiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiiijkllllllllllllllllllllllllllllllllllmmmmmmmmmmmmmmmmmmmmmmnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnnoooooooooooooooooooooooooooooooooooooooooooooooopppppppppppppppppppqrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrsssssssssssssssssssssssssssssssssssssssssssssssttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuuvvvwwwwwwwwwwxxxxxyyyyyyyyy
68. scala> 'a' to 'z' union ('A' to 'Z')
70. scala> "hello world" patch (6, "goof", 5)
71. res51: String = hello goof
73. scala> "hello world" updated(0,'H')
74. res54: String = Hello world

Yet more instances of '_'

A few more placeholder instances that I have remembered or was reminded of.

1. scala> class MyClass {             
2.      | var a:Int = _    // 1
3.      | def countDown = 10 to 5 by -1
4.      | }
5. defined class MyClass
7. scala> val obj = new MyClass()
8. obj: MyClass = MyClass@6ff0239
10. scala> val countDownMethod = obj.countDown _  // 2
11. countDownMethod: () => Range = < function>
13. scala> def multiple(a:Int)(b:Int) = a*b
14. multiple: (Int)(Int)Int
16. scala> val triple = multiple(3) _ // 3
17. triple: (Int) => Int = < function>
19. scala> List(1,2,3) foreach { _ => Console.println("Hello") } // 4
20. Hello
21. Hello
22. Hello

1. initialize a variable to its default value. If the = _ is left out then the definition will be an abstract declaration and will have to be defined in subclasses
2. create a reference to the method (rather than invoking the method and having a reference to the resulting value) (Thanks Daniel)
3. This is an example of currying; a new function is created with a single parameter.
4. where the underscore is used as an ignored and unnamed parameter (Thanks Alex)

What the @*!% is with the '_'

1. scala> import java.io._ // 1
2. import java.io._
4. scala> import java.io.{ File => _ } // 2
5. import java.io.{File=>_}
7. scala> object MyObj{ def count=(1 to 10) }
8. defined module MyObj
10. scala> import MyObj._ // 3
11. import MyObj._
13. scala> class MyClass { def countDown= 10 to 5 by -1} 
14. defined class MyClass
16. scala> val instance = new MyClass()
17. instance: MyClass = MyClass@69ebcd0
19. scala> import instance._ // 4
20. import instance._
22. scala> def multiply(by:Int,x:Int)=2*x
23. multiply: (Int,Int)Int
25. scala> val double=multiply(2, _:Int) // 5 
26. double: (Int) => Int 
28. scala> count foreach {i => double(i)}
30. scala> val double2:Int=>Int = multiply(2,_)
31. double2: (Int) => Int = & function>
34. scala> count foreach {i => double2(i)}
36. scala> count reduceLeft {_+_} // 6
37. res3: Int = 55
39. class Generic[T](val t:T)
41. val generic:Generic[_] = new Generic(2) // 7

While at a glance the underscores do not seem related, in fact the general rule is fill in the blank. Sca_la what goes in the blank?

Going through the examples:

1. import anything/everything in package
2. this seems to be an exception. Assign File to oblivion. (It is no longer imported)
3. import all methods in object
4. same thing. Import all methods in instance object
5. creat a new method where the first parameter is defined but all others will be defined later. In this case a special syntax is required so the compiler knows that a new method is desired and the missing parameters was not simply a mistake. There is another syntax for defining methods that do not require the _ for currying
6. reduceLeft takes method with 2 parameters. _ + _ creates a function of 2 parameters: the first '_' represents the first parameter and the second '_' represents the second parameter.
7. the '_' represents any value. This syntax is frowned upon since Scala has such a rich type system but can be useful when interoperating with Java. Perhaps a better declaration would be Generic[Any]