DSLisp

MyLisp is now named DSLisp – Domain Specific (Language) Lisp.

I have published it as a project on CodePlex:
http://www.codeplex.com/DSLisp

The new name imples the purpose of the project.
To act as a host for DSL’s.

The idea is to compile your DSL into the DSLisp AST.
And then run your DSL inside the DSLisp engine.
By doing this you can very easily add single step debugging and breakpoint support to your DSL. 

See the codeplex site for more info.

ObjectMapper 2008

The old ObjectMapper 2000 was and still is a great tool and I have used it in lots of projects to generate and map my entities.
But one thing that is clear, is that it was way way too complex for most users, it had more settings than the space shuttle and could be a real PITA in some cases.

So me and Mats have talked about adding mapping support inside VS.NET, much like the Linq to SQL mapping designer.
We haven’t actually decided exactly how it’s going to work yet, so there is not much to get excited about for now.

Anyway, I’ve started to build a visual class designer for it.
The control will be possible to use for any kind of Windows Forms project and will not be bound to Object Mapper.

Screenshots:

Main view and overview overlay:
classdesigner1.png

Zoom in and out:
classdesigner2.png

 Expand / Collapse:
classdesigner3.png

Custom styles:
classdesigner4.png

 I’ve also added some auto layout code to it, but it’s based on spring algorithm, which isn’t very good for class diagrams.

Well, if anyone is interested in some colaboration on this one let me know.
I figure that a decent UML’ish designer could be useful for more than us.
The plan is to release the designer as a standalone project anyway.

Adding Linq support to NPersist

I’m currently adding some real Linq support to NPersist.
I did start on it when the first public previews of Linq was released, but then I was kind of stuck in World of Warcraft for a while (OK, slightly more than a while).
But I’ve finally broken out of the world of Azeroth and I’m back in the real world again.

So, here is a few sneak peeks of what will be possible:

 var res = from cust in ctx.Repository<Customer>() 
          where (from order in cust.Orders 
                 where order.OrderDate == 
                 new DateTime(2008,01,01) && order.Total == 3.1 
                 select order).Count > 0 
          select cust;

 The Linq query will be turned into our own DSL NPath :

select * 
from Customer 
where ((( 
	select count(*) 
	from Orders 
         where ((OrderDate = #2008-01-01#) and (Total = 3.1)) 
        ) > 0))

The NPath query will then be transformed into SQL once the user tries to access the result.
Sure, it is some slight overhead to transform a 2nd query language, but the NPath parser is extremely fast, and NPersist use NPath internally so we just wanted to avoid making two SQL generators.

The SQL generation is by far the biggest beast inside NPersist, it’s big, furry and very angry, so we just want to stay away from it if we can. ;-)

We have also managed to solve the problem with Linq and Load spans that have haunted us since we first saw Linq.
“How the heck do we add load spans to the select when we cannot change the syntax of the select part?”
The solution is quite simple: “You don’t”
Instead, we pass the loadspan to the query source, like this:

var res = from cust in ctx.Repository(new LoadSpan<Customer> 
("Name","Email","Address.StreetName"))    

          select cust; 

This way we do not need to invent weird hacks that don’t really fit into the Linq syntax.
The solution might have been obvious for others, but we were really stuck on it, pretty much because we have always specified load spans in the select clause of NPath.

Optimizing MyLisp

Today I stumbled across the weirdest thing.

I was comparing the performance of MyLisp to L# and MyLisp was terribly slow in comparison.
(Running the exact same Fibonacci code)

So I checked the code of L# and they seem to do pretty much the same as I do, quite similar design and code.

The only thing I could find was that I evaluated the first arg twice when calling Add, Sub,Mul and Div.
So I changed the code for those functions so that all args are evaluated only once.
And suddenly the performance was increased by some 1000 times or so, things that took a bout a minute before now completes in less than 0.1 sec.
I would get it if performance was increased about 2 times, but not like this.
The only thing I can think of is that the original code did some bugged recursive evaluation of some sort.

Well, I’m happy about the performance gain, but a bit annoyed that I can’t find the reason why the old code was so much slower.

Updated source code (C# 3) can be downloaded at:
http://www.puzzleframework.com/roger/mylisp.zip

[Edit. oh dear god I’m so stupid..]

Two minutes after I published this post I found the cause.
I even wrote the reason in this post..
I evaluated the first arg in Add, Sub, Mul, Div twice.
And the Fib code looks like this:

(= fibonacci (fn (n) 
   (if (< n 2) n 
      (+ (fibonacci (- n 1)) (fibonacci (- n 2)))))) 
;;;         ^ problem is right there...

The first arg of the last Add call is a recursive call to itself..
I feel so dumb now..
Well I guess the best way to find the cause of a problem is to try to describe it for someone else. 

In this case it was pretty much like that IQ test where you should count the letter “F”‘s in an english text where most people don’t count the “F” in “of”.

I was only looking at the (- n 1) and (-n 2) parts of the code..
Not seeing that I was also adding the result of recursive function calls.

Genetic programming / Math

For those of you who are interested in evolution / GP.
Here is a small app that I made to reverse engineer blackboxed formulas.

The application is fed with a “problem domain”

The problem domain consists of multiple cases, like this:

Case1:
When X is 1 and Y is 2 I want the result to be 3
Case2:
When X is 3 and Y is 5 I want the result to be 8
Case3:
When X is 10 and Y is 10 I want the result to be 20

That was a very simple problem.
And the formula to solve the problem would be (X+Y).

Ofcourse the problems aren’t described in english, it’s code. but this is the idea behind it anyway.

Once the application is fed with a problem domain, it will create a population of “formulas”
Each formula have its own DNA, the DNA in this case is the various functions, constants and variables.

Each formula in the population will be fed with the data from each problem case and then evaluated.
Depending on how close to the goal the result is, the better the error level is.

Once each problem case have been evaluated by a formula, all the error levels per case will be summed up.
The summed error level is then used to decide what formulas are allowed to survive and breed.
(There is also a bit of crossover going on between formulas, but I won’t get into that now.)

So how fancy stuff can this application solve?

Well, here are some examples:
Problem data and result is shown on the pictures.

And no, it does not care if the result is pretty, it just strives to find a working formula.
ofcourse this could be altered so that once the solution is found, it continues to evolve and promote shorter formulas.

GenMath sample 1

GenMath sample 2

GenMath sample 3

Complex solution, the application generated a formula to convert decimal 0-15 into binary representation:

genmath41.gif

After some optimization of the final formula it came up with this:

((((int)(((((100 * ((int)(-8) & (int)(x))) - (-23 * ((int)(-4) & (int)(x)))) + x) - 2)) | (int)(7)) + 1) + x)

Not the prettiest formula in the world, but it does what it is supposed to..
Also consider that the extensive use of “(int)” is not really generated by the application,
Its my ToString implementation of binary operators that add those.

If we remove all the extra clutter which is really only part of the visual representation, we would end up with this:

(((100 * (-8 & x) - (-23 * (-4 & x)) + x) - 2) | 7) + 1 + x;

I find it quite interesting to see how evolution solves the problem, it’s far from how a human would have solved it.
Evolution also have a habit to cheat, and very much so.

Eg, if I allow the application to use the XOR binary operator, it sometimes finds XOR patterns that can be applied to the problem cases, and thus solving the problem, but not the way you might have wanted it to..

Anyway, for those interested:
The application source code (C# 2) can be downloaded here:
Genetic Math

MyLisp: Callstack and threading

I’ve added multi threading support to MyLisp today.

The earlier implementation of the callstack was not up to the task, so I had to rewrite it completely.
The new callstack is based on stack frames which can hold local symbols.

So behold! :-) , the first multi threaded application in MyLisp

(func FooBar () 
  ( 
    (let i 0 
      (for i 0 1000000 
        (print (format "thread {0}" i)))))) 

(= thread (new-thread FooBar)) 
(call thread Start) 
(for i 0 1000000 
  (print (format "main {0}" i)))

Sure , the “new-thread” function is a bit of cheating, I dont have any generic code for .NET delegate <-> MyLisp delegate yet.
So I have to use hard coded methods to cast from and to delegates for now.

OK, this was not much of a real post, more of a “yay, it works!” shout :-)