Scala-IO Core: Output

The Output object is the primary trait for writing data to a resource. The basic usage is very simple but can get more complex when one wishes to serialize objects.

Lets start with the basic usage: A common need is to write several times to a single Output without overwriting the data. To do this one can use the processing API. A future post(s) will look at the processing API in more detail but for now a simple example:

Scala-IO Core: Long Traversable

The LongTraversable trait is one of the most important objects in Scala IO. Input provides a uniform way of creating views on the data (as a string or byte array or LongTraversable of something like bytes.)

LongTraversable is a scala.collection.Traversable with some extra capabilities. A few of the salient points of LongTraversable are:

• It is a lazy/non-strict collection similar to Stream. In other words, you can perform operations like map, flatmap, filter, collect, etc... without accessing the resource
• Methods like slice and drop will (if possible for the resource) skip the dropped bytes without reading them
• Each usage of the LongTraversable will typically open and close the underlying resource.
• Has methods that one typically finds in Seq.  For example: zip, apply, containsSlice
• Has methods that take or return Longs instead of Ints like ldrop, lslice, ltake, lsize
• Has limitFold method that allows fold like behaviour with extra features like skip and early termination
• Can be converted to an AsyncLongTraversable which has methods that return Futures instead and won't block the program
• Can be converted to a Process object for advanced data processing pipelines

Example usage:

The limitFold method can be quite useful to process only a portion of the file if you don't know ahead of time what the indices of the portion are:

Scala-IO Core: Resource, Input

Just a note: all these examples have been tested in REPL so go ahead and fire up the sbt console in the example project and try these out.

Resource

Resource is the fundamental component of Scala-IO. A Resource is essentially anything that has a simple open/close lifecycle. The Resource trait handles the lifecycle for the developer allowing him to focus on the IO logic.

In the typical use-case one of the Resource subclasses will be used. They are more useful in general because they will have one of higher level traits mixed in like Input or Output.

The most typical way to create a Resource is with the Resource object which is a factory method for creating Resource objects from various types of Java objects.

While Resource is the foundation Trait, Input and Output are the Traits most commonly used, The user-facing traits if you will.

Here are a few examples of creating Resources: There are advanced usages of Resource that we will get into in later posts. At the moment I want to focus on Input, Output and Seekable Traits. In later posts we will look at how to integrate with legacy Java APIs and how to access the underlying resource using the loan pattern.

Input

The Input Trait provides methods for accessing the data of the underlying resource in various different way. As bytes, strings, lines, etc...

There are two basic types of methods. Methods that return LongTraversable objects and methods that load the entire Resource into memory. For example: string and byteArray load the entire resource into memory while bytes and chars return a LongTraversable.

What is a LongTraversable? That will be the next post :-). Summarized, it is a specialized Lazy/non-strict Traversable.

Scala-IO Getting Started

For the next several posts you will need to have Scala-IO installed and probably should have a sbt project as well.

There are currently 2 Scala-IO 0.4 releases.

• Scala-io 0.4-seq - A version of Scala 0.4 without the Akka dependency and therefore no ASync support
• Scala-io 0.4 - The full version that contains an Akka  dependency

The Scala 2.10 versions will have no Akka dependency but can optionally use Akka.

So getting started:

Download the example project on the docs website (http://jesseeichar.github.com/scala-io-doc/latest):

• Go to Getting Started and follow instructions for downloading and running the example project.  The following goes through the steps for the 0.4.1 instructions.

The last line (Right(770)) is not a command to enter; it is the result of the asynchonous call.

Introducing Scala-IO

This is the start of a series of posts on Scala-IO.  Scala-IO is as the name implies a library for performing input and output operations with Scala.  There are 4 main facets to the library

• Basic IO - Reading and writing to some underlying resource.  The current implementation is Java based and thus allows reading and writing to resources like java.io.Readers, Writers, Channels, Streams, etc...
• File API - A library loosely designed after java 7 nio.file API with an additional simple unix like DSL for traversing and searching the filesystem.  It is a pluggable architecture which allows plugins for systems like WebDav or Zip filesystems to be addressed in a similar manner as the local filesystem.  The included implementation is for the local file system and is implemented on the java.io APIs
• Asynchronous Access - Throughout the APIs are both synchronous and asynchronous options allowing both models of programming to be easily used.  
• In the 2.10.x + versions the future implementations are pluggable but require no additional libraries if so that is the desire
• In 2.9.x versions there are two different dependencies one with asynchronous APIs implemented on Akka and one without any asynchronous APIs
• Processor API - An API for defining complex IO processes declaratively.

This series will look at normally a small and simple IO operation each day (or so) rather than only a few in-depth articles.  This is required because of my limited available time.

With the introduction done lets look at two small examples:

Read File with Core API (not File API):

Same thing but with File API:

zipWithIndex

A common desire is to have access to the index of an element when using collection methods like foreach, filter, foldLeft/Right, etc... Fortunately there is a simple way.

List('a','b','c','d').zipWithIndex.

But wait!

Does that not trigger an extra iteration through the collection?. Indeed it does and that is where Views help.

List('a','b','c','d').view.zipWithIndex

When using a view the collection is only traversed when required so there is no performance loss.

Here are some examples of zipWithIndex:

1. scala> val list = List('a','b','c','d')
2. list: List[Char] = List(a, b, c, d)
4. /*
5. I like to use functions constructed with case statements
6. in order to clearly label the index.  The alternative is 
7. to use x._2 for the index and x._1 for the value
8. */
9. scala> list.view.zipWithIndex foreach {case (value,index) => println(value,index)}
10. (a,0)
11. (b,1)
12. (c,2)
13. (d,3)
15. // alternative syntax without case statement
16. scala> list.view.zipWithIndex foreach {e => println(e._1,e._2)}
17. (a,0)
18. (b,1)
19. (c,2)
20. (d,3)
22. /*
23. Fold left and right functions have 2 parameters (accumulator, nextValue) 
24. using a case statement allows you to expand that but watch the brackets!
25. */
26. scala> (list.view.zipWithIndex foldLeft 0) {case (acc,(value,index)) => acc + value.toInt + index} 
27. res14: Int = 400
29. // alternative syntax without case statement
30. scala> (list.view.zipWithIndex foldLeft 0) {(acc,e) => acc + e._1.toInt + e._2} 
31. res23: Int = 400
33. /*
34. alternative foldLeft operator.  The thing I like about this
35. syntax is that it has the initial accumulator value on the left 
36. in the same position as the accumulator parameter in the function.
38. The other thing I like about it is that visually you can see that it starts with
39. "" and the folds left
40. */
41. scala> ("" /: list.view.zipWithIndex) {                          
42.      | case (acc, (value, index)) if index % 2 == 0 => acc + value
43.      | case (acc, _) => acc                                       
44.      | }
45. res15: java.lang.String = ac
47. /*
48. This example filters based on the index then uses map to remove the index
49. force simply forces the view to be processed.  (I love these collections!)
50. */
51. scala> list.view.zipWithIndex.filter { _._2 % 2 == 0 }.map { _._1}.force
52. res29: Seq[Char] = List(a, c)

Return value of a block

A common misunderstanding is that a code block (without parameters) is a function. That is not the case. A code block is a sequence of statements that are executed and result the last statement is returned. That sounds like a Function0, however, if the block is passed to a method/function only the last statement will be returned to the function/method. If that method/function expects a function as the parameter the last statement maybe returned as a function not a value, this means that the block itself is not a function.

1. scala> var count = 0                                                                                                                         
2. count: Int = 0
4. // the last statement is returned as a function so count
5. // is incremented only one during the creation of the function
6. scala> List(1,2,3,4).map{count += 1;_ + 1}
7. res9: List[Int] = List(2, 3, 4, 5)
9. scala> count
10. res10: Int = 1
12. // now the count increment is within the function
13. scala> List(1,2,3,4).map{i => count += 1;i + 1}
14. res11: List[Int] = List(2, 3, 4, 5)
16. scala> count
17. res12: Int = 5

The previous example demonstrates a Gotcha if I ever saw one. Map expects a function so the block essentially constructs a function. The last statement being the function. The first line count += 1 executed only once because it is part of creating the function not part of the resulting function. This is equivalent to:

1. scala> val x = {count += 1 ; i:Int => i +1}
2. x: (Int) => Int = < function1>
4. scala> List(1,2,3,4).map(x)
5. res15: List[Int] = List(2, 3, 4, 5)

Beginning a block with the parameter list signals that the entire block is a function.

Rule of thumb: Functions with placeholder parameters should be a single statement.