The beauty of clean syntax and deep abstraction is an often overlooked feature of functional programming.  As they say, people come to functional programming for the concurrency but stay for the beautiful code (actually, I just made that up).  Also included: POPL 2009, S#arp and functional unit testing.

 

Blog - Matthew Podwysocki’s Series on Functional Unit Testing

In this eight part series Matthew covers unit testing in Haskell from the basics setting up HUnit to the details of writing purely functional tests.

This series is the only place where I’ve seen functional programming unit testing talked about with any significant depth.  While in writing this series Matthew used Haskell, it and F# share much in common and it’s obvious that he worked hard to make his posts apply to both.  I consider it a must read for anyone thinking about taking functional software engineering seriously. 

Although he has finished the series, I hope that in the future he takes some time to focus on unit testing in F# specifically.  I’ve yet to see topics such as mocking or ensuring referential transparency by test covered.

 

Transcription - POPL 2009 Grand Challenges Panel Summary 

This panel was lead by such research giants as Simon Peyton Jones, Xaveir Leroy, Kathryn McKinley, Greg Morrisett and Arvind.  In it they discuss where programming language development is heading and the factors driving us there.  Well worth the read for anyone interested in programming language evolution.

 

Blog - Mark Needham’s F# vs C# vs Java: Collection Parameters

In this post Mark compares list operations in F#, C# and Java.  It’s striking to see how far we have come in terms of clean, concise syntax.  Actually, it seems kind of reminiscent of evolution of man posters.

 

Blog - Jafar Husain’s F#: Real Sharp

When working in F# over a long period it’s easy to forget that one of the best things about it is how it’s concise syntax and deep abstractions make your code so much easier to understand and beautiful to look at.  In his post Jafar reminds us of this by contrasting C# with F# samples.

 

Software – S#arp Architecture Beta 1.0 Available

S#arp Architecture promises to make building web applications much easier by making much of the glue we currently use in ASP.NET web applications obsolete. 

I am a big fan of inversion of control and it is something that comes very natural in functional programming languages.  I hope to see some examples with S#arp Architecture and F# internals at some point in the near future.

The beauty of clean syntax and deep abstraction is an often overlooked feature of functional programming.  As they say, people come to functional programming for the concurrency but stay for the beautiful code (actually, I just made that up).  Also included: POPL 2009, S#arp and functional unit testing.

 

Blog - Matthew Podwysocki’s Series on Functional Unit Testing

In this eight part series Matthew covers unit testing in Haskell from the basics setting up HUnit to the details of writing purely functional tests.

This series is the only place where I’ve seen functional programming unit testing talked about with any significant depth.  While in writing this series Matthew used Haskell, it and F# share much in common and it’s obvious that he worked hard to make his posts apply to both.  I consider it a must read for anyone thinking about taking functional software engineering seriously. 

Although he has finished the series, I hope that in the future he takes some time to focus on unit testing in F# specifically.  I’ve yet to see topics such as mocking or ensuring referential transparency by test covered.

 

Transcription - POPL 2009 Grand Challenges Panel Summary 

This panel was lead by such research giants as Simon Peyton Jones, Xaveir Leroy, Kathryn McKinley, Greg Morrisett and Arvind.  In it they discuss where programming language development is heading and the factors driving us there.  Well worth the read for anyone interested in programming language evolution.

 

Blog - Mark Needham’s F# vs C# vs Java: Collection Parameters

In this post Mark compares list operations in F#, C# and Java.  It’s striking to see how far we have come in terms of clean, concise syntax.  Actually, it seems kind of reminiscent of evolution of man posters.

 

Blog - Jafar Husain’s F#: Real Sharp

When working in F# over a long period it’s easy to forget that one of the best things about it is how it’s concise syntax and deep abstractions make your code so much easier to understand and beautiful to look at.  In his post Jafar reminds us of this by contrasting C# with F# samples.

 

Software – S#arp Architecture Beta 1.0 Available

S#arp Architecture promises to make building web applications much easier by making much of the glue we currently use in ASP.NET web applications obsolete. 

I am a big fan of inversion of control and it is something that comes very natural in functional programming languages.  I hope to see some examples with S#arp Architecture and F# internals at some point in the near future.

In the image processing world, like most computational problems, we often think our work is composed of only two basic ideas: representation and transformation.  Of course, one may have many layers of both representations of transformations and transformations of representations which can make things appear quite complex at times.

However,  the problem is much more simple than it appears.  This is because a representation can be considered as a transformation from a zero or identity state.  Thus, in writing a symbolic language for image processing, we are left with only a single idea to consider:  transformations.  By composting layers of transformations we can apply image processing techniques in way which is not only bidirectional and platform agnostic but also comes along with a host of other benefits.

 

Let us consider a simplified example of processing an image:

1) We read in a file (representation) and use a codec (transformation) to convert it into a format understood by our API (representation).

2) We then perform some type of algorithm on that data (transformation) which results in some type of output (representation).

3) Finally, via another codec (transformation), another file is saved to disk (representation).

 

In most cases there are a great number of intermediate representations.  Each is a full copy of the previous iteration with whatever changes have been so far applied.  Essentially, the same information is copied over and over again in memory.  We do allow for some kinds of in-place processing, however, this is bad as when the operation has been completed, the previous representation has been destroyed.

 

Instead, what if we batched up sets of transformations?  This could have many benefits:

1) The most obvious benefit is that of parallelization.  Even at the simplest level of functional composition, these transformations could be handed off to a cluster for asynchronous processing or saved for a later batch processing job.

2) With an intermediate symbolic transformation language, processing algorithms could potentially be combined and reduced to produce a single transformation out of many.  This would significantly reduce the processing overhead as well as the number of intermediate memory representations.

3) An intermediate symbolic language which encompassed both codec and processing may make it possible to push the processing transformation through the codec transformation and in so doing no longer need to have any intermediate memory representation.  This could provide significant memory and processing speed time benefit. 

4) The intermediate symbolic language could be saved into the files themselves thus removing the need for the codec to be present on the end machine.  Admittedly, the user would also need the image language interpreter.

5) Instead of applying simple image processing algorithms to an image, the symbolic representation could be appended to the end of the file.  This would be quite similar to layers in practice.  In this way it would be possible to view the image at all stages of transformation.

6) For large or proprietary transformations, the representation could be kept on the internet and either be downloaded or, in the case where the owner did not want to expose their algorithm, a flattened representation could be sent out and a processing delta could be sent back.

 

Conclusion

Of course, when I speak of data I don’t only mean the image itself.  This technique could also be applied to many classes of data or algorithm.  Most notably for us, image metadata.

My initial goal is to build a basic codec representation along with some simple transformations.  Currently, I am researching bidirectional, reversible and declarative languages as examples.  With F# as a base language I believe it will be possible to build something portable to other ML variants.

In the image processing world, like most computational problems, we often think our work is composed of only two basic ideas: representation and transformation.  Of course, one may have many layers of both representations of transformations and transformations of representations which can make things appear quite complex at times.

However,  the problem is much more simple than it appears.  This is because a representation can be considered as a transformation from a zero or identity state.  Thus, in writing a symbolic language for image processing, we are left with only a single idea to consider:  transformations.  By composting layers of transformations we can apply image processing techniques in way which is not only bidirectional and platform agnostic but also comes along with a host of other benefits.

 

Let us consider a simplified example of processing an image:

1) We read in a file (representation) and use a codec (transformation) to convert it into a format understood by our API (representation).

2) We then perform some type of algorithm on that data (transformation) which results in some type of output (representation).

3) Finally, via another codec (transformation), another file is saved to disk (representation).

 

In most cases there are a great number of intermediate representations.  Each is a full copy of the previous iteration with whatever changes have been so far applied.  Essentially, the same information is copied over and over again in memory.  We do allow for some kinds of in-place processing, however, this is bad as when the operation has been completed, the previous representation has been destroyed.

 

Instead, what if we batched up sets of transformations?  This could have many benefits:

1) The most obvious benefit is that of parallelization.  Even at the simplest level of functional composition, these transformations could be handed off to a cluster for asynchronous processing or saved for a later batch processing job.

2) With an intermediate symbolic transformation language, processing algorithms could potentially be combined and reduced to produce a single transformation out of many.  This would significantly reduce the processing overhead as well as the number of intermediate memory representations.

3) An intermediate symbolic language which encompassed both codec and processing may make it possible to push the processing transformation through the codec transformation and in so doing no longer need to have any intermediate memory representation.  This could provide significant memory and processing speed time benefit. 

4) The intermediate symbolic language could be saved into the files themselves thus removing the need for the codec to be present on the end machine.  Admittedly, the user would also need the image language interpreter.

5) Instead of applying simple image processing algorithms to an image, the symbolic representation could be appended to the end of the file.  This would be quite similar to layers in practice.  In this way it would be possible to view the image at all stages of transformation.

6) For large or proprietary transformations, the representation could be kept on the internet and either be downloaded or, in the case where the owner did not want to expose their algorithm, a flattened representation could be sent out and a processing delta could be sent back.

 

Conclusion

Of course, when I speak of data I don’t only mean the image itself.  This technique could also be applied to many classes of data or algorithm.  Most notably for us, image metadata.

My initial goal is to build a basic codec representation along with some simple transformations.  Currently, I am researching bidirectional, reversible and declarative languages as examples.  With F# as a base language I believe it will be possible to build something portable to other ML variants.