More Linq support for NPersist

I’ve added a bit more Linq support for NPersist today.

The nice thing is that we can “cheat” in our Linq provider, we can transform our Linq queries into NPath queries.
Thus, I don’t have to touch the wicked SQL generation.
I just have to produce valid NPath, which is pretty similair to Linq.

Just look at the following code:

private string ConvertAnyExpression(MethodCallExpression expression) 
    string fromWhere = ConvertExpression(expression.Arguments[0]);     
    if (expression.Arguments.Count == 1) 
        return string.Format("(select count(*) from {0}) > 0", fromWhere); 
        LambdaExpression lambda = expression.Arguments[1] as LambdaExpression; 
        string anyCond = ConvertExpression(lambda.Body); 
        return string.Format("(select count(*) from {0} and {1}) > 0", fromWhere,anyCond); 

Thats all the code that we needed in order to support “list.Any()” and “list.Any(x => ….)”
This both makes the Linq provider easier to build, and it makes it way easier to unit test it too.
We simply have to compare the result with an expected NPath string.
Like this:

string expected = "select * from Customer where ((Customer != ?))"; 
string actual = res.Query.ToNPath(); 
Assert.AreEqual<string>(expected, actual);

So I think we have most of the standard query elements in place now.
I just have to add support for a few more things inside NPath, things like “skip”, “is” etc.

Lazy evaluation

My Lisp now supports Lazy evaluation.

I’ve made it possible to define per function if it should use lazy or eager evaluation.
(The next step will be to decide that through code analysis.)

For those who don’t know what lazy evaluation is about, the purpose of lazy evaluation is to avoiding unnecessary calculations.

Take the following example (in C#)

public static void FooBar () 
  int res = GetSomeValue();         
  //pass the value to a logger 
  Logger.Log ("value of GetSomeValue was:" , res); 

public static int GetSomeValue() 
 //some realy heavy algorithm here 
 ... heavy code ... 
 ... more heavy code ...   
 return res; 

... logger code ...           

public void Log (string message,params object[] args) 
 //lets assume we can attach multiple providers to our logger 
 foreach (LogProvider provider in LogProviders) 

In this case, we would always need to execute “GetSomeValue” first in order to call our logger.
EVEN if there is no provider attached to the logger.
So, even if we don’t want to write anything to disc or DB or anywhere, we would still have to call “GetSomeValue”.

It is ofcourse possible to add special code to your consumer , “if (logger.Providers.Count == 0) then ignore…”.
But that forces you to add responsibility to your code that isn’t supposed to be there.
My FooBar method should not have to care about the number of log providers etc.

Lazy evaluation can solve this for us.
By applying Lazy evaluation to “GetSomeValue” method, we would only need to call the method once the result of it is used.
Sounds odd?
How can you use the result before the code have executed?

It’s quite simple, its done through delegates, we can even do this in C# by passing delegates around.
However, you do have to know that you are passing delegates and not simple values, so it is not very implicit in C#.

In My Lisp, the code would look something like this:

(func FooBar () 
        (= res (GetSomeValue)) 
        (call logger log "value of GetSomeValue was:" res)))     

(lazy-func GetSomeValue () 
         ... heavy code ... 
         ... more heavy code ... 
        (return res)))     


(func Log (message data) 
        (foreach provider Providers     

            ;;; GetSomeValue will be called here 
            (call provider WriteString message data))))

As you see, there is no special code to handle the lazy evaluation.
The only thing that I need to do was define “GetSomeValue” as a lazy function.

So if we do not have any log providers attached to our logger, “GetSomeValue” will never be executed, since its result is never needed.

And if there are providers attached, “GetSomeValue” will be executed from within the foreach loop inside the logger.
(Once, then caching its value)

Pretty slick IMO.

Lisp weekend

I’ve been reading up a bit on functional programming the last few week, the reason is just to comprehend the new features and possibilities in .NET 3.5 as much as possible.

Anyway, I got a bit carried away and started to read about Lisp, and decided to learn what it’s all about.
So what better way to learn a language than to make your own parser for it is there? ;-)

I started to hammer away on a simple parser, and once the parser was done, I couldn’t stop, so I began writing an engine too.
So after a few hours of Aha moments, I finally got my very own Lisp(ish) code executor and a bit more understanding for the language. ;-)

Well, enough blabbering, here are a few samples of whats currently possible in my still un-named language.

Hello world:

(print 'Hello world!')

Simple function and call:

(defun Mul (x y) 
 (* x y))       

(print (Mul 2 3))


(let my-var 'hello lisp') 
(let my-int 123) 
(let my-double 123.456) 
(let half-pi (/ pi 2)) 
(let my-arr (arr 1 2 3 4 5 6 7))


(foreach item my-arr 
 (print item))      

(for i 1 20 
 (print i))      

(let i 0) 
(while (< i 20) 
 ((print i) (++ i)))


(let my-lambda (lambda (x y) (* x y))) 
(my-lambda 2 3)


(let my-delegate Mul) // delegate to Mul 
(let print other-print-func) //redirect the print function to "other-print-func"


(let form (new Form)) 
(set form Text 'hello windows forms') 
(let button (new Button)) 
(set button Text 'my button') 
(set-event button Click MyButtonClick) 
(list-add (get form Controls) button) 
(call form Show)

List comprehensions:

(foreach item (select (lambda (concat 'transformed: ' item '!')) 
              (where (lambda (> (get item Length) 3)) 
              (list 'foo' 'bar' 'roger' '.net' 'lisp')))       

      (print item))

The next step will be to make it possible to define your own classes.
Im thinking of emitting true .NET classes and let the methods redirect the calls to the engine.
Thus making it possible to redefine the behaviour of a method in runtime.

That, and find some reason to use it :-P

Making a custom Linq engine

You can download the complete source here
NOTE: The download contains much more than this post.

Today I’m going to show how to make a custom linq engine.

So, why would you want to create your own engine?

Well replicating the standard in mem engine is pretty pointless except for educational purposes.
But you might need custom engines for your own frameworks where you want to query your own special DB or media of some sort.
We use this in NPerist, we transform the linq query into our own NPath DSL and then run the NPath query through NPersist into the database.

Anyway, back to the sample.
Lets start by creating a simple console application and add the following code in the Main method:

using System.Linq; 
//setup test data 
List<string> source = new List<string>() { 
    "protected internal", 

//run a simple query 
var result = from item in source 
             where item.Length > 1 
             select item + " hello linq"; //display the result                

foreach (var item in result) 
    Console.WriteLine("{0}", item); 

Nothing fancy, we simply init a list with a few strings, and then we run a standard linq query on it.

Now lets start writing our own Linq engine.
To do this we need a few delegates:

// T = item type 
// IN = item type IN 
// OUT = item type OUT  
//delegate to handle the where clause 
public delegate bool WhereFunc<T>(T type);                

//delegate to handle the select clause 
public delegate OUT SelectFunc<IN, OUT>(IN item);

We also need two extension methods:

namespace MyLinqEngine; 
public static class Sequence 
    //extension method that will handle where clauses 
    public static List<T> Where<T> 
    (this List<T> source, WhereFunc<T> matchDelegate) 
        //create a result list 
        List<T> result = new List<T>();         
        //scan all items in the source 
        foreach (T item in source) 
            //check for match 
            bool match = matchDelegate(item);             
            if (match) 
                //add match to result 
        return result; 
    //extension method that will handle select clauses 
    public static List<OUT> Select<IN, OUT> 
    (this List<IN> source, SelectFunc<IN, OUT> selectorDelegate ) 
        //create a result list 
        //since the result will have the exact same size as 
        //the source we can init it with a fixed size 
        List<OUT> result = new List<OUT>(source.Count);                  
        //scan each item in the source 
        foreach (IN item in source) 
            //transform the items in the source to an 
            //item of the given out type 
            OUT resultItem = selectorDelegate(item); 
        return result; 

And that’s it!
That’s all you need in order to create the simplest form of custom Linq engine.

The default Linq extensions are activated by the “using System.Linq;” at the top of your program.cs class file.
We can now remove that line and replace it with “using MyLinqEngine;” in order to activate our own linq extensions.

You’ve now got your very own debuggable Linq engine.

Happy coding.


You can download the complete source here
NOTE: The download contains much more than this post.