Writing a Simple DSL Compiler with Delphi [4. Parser]

This article provides a description of a parser used in my toy language project. If you are new to this series, I would recommend to start reading with this post.

Please note that this article describes an initial implementation of the parser. If you want to browse the code while reading the article, make sure that you have switched to branch dsl_v1.

After a short break I'm returning to my "toy compiler" series. This time I'll describe the working of the parser - the part of the code which reads the input (usually in a tokenized form) and generates internal representation of the program (in my case an abstract syntax tree - AST).

The goal of my project was to study the compilation step and parser was just a necessary evil that I had to deal with. That's why it is written in a pretty primitive manner, without using any improvements like a Pratt parser.

Delphi and Linux

I'd just like to remind my Slovenian readers that on 28th this month I'll be having a presentation about RAD Studio and Linux in Ljubljana.

As the presentation will be given in Slovenian language, the rest of my post containing the description of the presentation is written in that language, too.

Vljudno vabljeni na delavnico "RAD Studio in Linux", na kateri si bomo ogledali:

• Kako namestiti Ubuntu v virtualni računalnik.
• Kaj storiti, ko namestitev nagaja.
• Kako pripraviti virtualni računalnik in RAD Studio za delo.
• Kako napisati konzolno aplikacijo za Windows in Linux, ki streže podatke z uporabo tehnologij FireDAC in DataSnap.
• Kako napisati grafično aplikacijo za Windows, OS X, iOS, Android, ki prikazuje in spreminja podatke na aplikacijskem strežniku iz prejšnjega koraka.
• Z nekaj sreče pa še: 
• Kako to grafično aplikacijo pognati na Linuxu in
• Kako konzolno aplikacijo spremeniti v pravi Linux "service".

Vstop prost, prosimo vas le, da se vnaprej registrirate, da bomo znali pripraviti zadostno količino kave in prigrizkov ;)

Autumn is coming ...

... and with it, a whole bunch of events.

First, I'll be going to IBC 2017 in Amsterdam where we are presenting our products in Hall 2. I'll be there from 15th to 18th, so if anybody wants to meet, contact me.

Immediately after that I'm going to Zegrze (just north of Warsaw) to speak at Zlot Programistów Delphi (Delphi programmers convention) on 21st and 22nd. Never been there before, should be great fun.

A week later (28th) I'll be leading a workshop on Delphi and Linux in Ljubljana. (Slovenian readers - you can sign on now.)

To finish it up, I'll present two all-new topics in ITDevCon in Rome on October 11th and 12th.

All that in four weeks. Should be quite an interesting time ;)

Introducing OmniThreadLibrary Core

Yesterday I wanted to use OmniThreadLibrary as a git submodule in a top secret ;) open source project I'm working on and I was a bit shocked when it turned out that newly cloned OmniThreadLibrary folder is a 83 megs in size. Given that I only need it to support my other project and that I won't do any OTL fixing/development in this submodule, that looked a bit excessive.

So I went ahead and created a core branch which contains only the barebones - the OmniThreadLibrary root folder without any subfolders. If you want to compile such submodule, you'll also need the GpDelphiUnits repo. Together they weight measly 3,5 MB, which is a big improvement over the original 80+ megs.

I'll be keeping core in sync with the latest OTL release, not with the HEAD, so it will also provide a stable platform for all depending repositories.

Writing a Simple DSL Compiler with Delphi [3. Tokenizer]

This article provides a description of a tokenizer used in my toy language project. If you are new to this series, I would recommend to start reading with this post.

Please note that this article describes an initial implementation of the tokenizer. If you want to browse the code while reading the article, make sure that you have switched to branch dsl_v1.

With this article I'm moving into the gritty part of the project - the code which reads the source code and turns it into a beautiful abstract syntax tree. In other words, I'll be talking about the parser.

I must admit that I spent as little time on the parser as I could. After all, my main goal was to convert AST into an executable code, not to parse input. Still, one cannot write a compiler without writing a parser, so ... here I am.

Writing a Simple DSL Compiler with Delphi [2. Abstract Syntax Tree]

This article provides a description of an abstract syntax tree used to represent "The Language". If you are new to this series, I would recommend to start reading with this post. 

Please note that this article describes an initial implementation of the AST. If you want to browse the code while reading the article, make sure that you have switched to branch dsl_v1.

An abstract syntax tree (AST) is, simply put, a symbolic representation of the program in a form of a tree.

While the textual representation of a program is great for us, humans, computers have hard time dealing with it. Because of that, a special part of any interpreter/compiler, called parser, reads the input and converts it into a computer-readable format - AST. This tree can then be used for multiple purposes. We can, for example, feed to into an interpreter which will then run the program for us, or we can feed it into a compiler to generate an executable program or cross-compiler to generate an equivalent program in a different programming language.

Writing a Simple DSL Compiler with Delphi [1. The Language]

Part 1: The Language

This article provides an informal definition of a simple language (a.k.a. "The Language") I am writing a compiler for. If you are new to this series, I would recommend to start reading with this post.

Let's start with a simple example which calculates an i-th Fibonacci number.

 fib(i) {
   if i < 3 {
     return 1
   } else {
     return fib(i-2) + fib(i-1)
   }
 }

The code is quite simple. First two numbers in a Fibonacci sequence are 1 and every other Fibonacci number is a sum of previous two in the sequence.

Writing a Simple DSL Compiler with Delphi [0. Introduction]

Part 0: Introduction

Some time ago I was listening to the Hanselminutes podcast where a guy described how he wrote a simple interpreter in Go language (YOU should write an interpreter with Thorsten Ball). I was not really interested in writing an interpreter - I did that a long time ago - but a thought crossed my mind. I asked myself whether I can do something better - write a compiler. (Or, rather, a kinda-compiler. You'll see.)

What I wanted to do is to take a small language, parse it, generate an abstract syntax tree, and then convert this into one large anonymous function calling other anonymous functions calling other anonymous functions and so on and so on. It is hard to explain, so let me try using a very simple example.

OmniThreadLibrary 3.07.3

TL;DR: Update OmniThreadLibrary now!

A nasty bug was found in the DSiWin32 library. It causes the DSiTimeGetTime64 function to work incorrectly when called from multiple threads at the same time. As this function is central to time measurement in the OmniThreadLibrary, it was essential to release new, fixed version.

This version also contains two small enhancements.

• SetTimer method now accepts TProc and TProc timer methods.
• IOmniTask implements method InvokeOnSelf which can be used to schedule anonymous function execution from a task back to self.  

As usual, you can get OmniThreadLibrary from GitHub (3.07.3, HEAD), download the zip, install it with GetIt or with Delphinus.

Damn, multithreading is hard!

OmniThreadLibrary 3.07.2

This is just a small update which adds few helpful methods. Although the change log mentions a potentially breaking change I don't think this will affect anybody. (And if you are not sure, you can enabled backward-compatible behaviour by adding one line to the code.)

I have also (finally) found some time to work on The book. It is now fully up to date (all OmniThreadLibrary features are documented on its whooping 293 pages) and is only missing one or two introductory chapters.

As usual, you can get OmniThreadLibrary from GitHub (3.07.2, HEAD), download the zip, install it with GetIt or with Delphinus.

Change log follows after the break.

RAD Studio 10.2 links

This post contains just a bunch of links to online resources that I mentioned on the today’s presentation.

10.2 Tokyo comes to Slovenia

In two days I’ll present RAD Studio 10.2 Tokyo in Ljubljana.

Click here for more information and to register.

OmniThreadLibrary 3.07.1

This update brings only few small changes. The biggest of them is support for the Delphi 10.2 Tokyo.

You can get it as a zipped download, git checkout, via Delphinus, and via GetIt.

For more information on OmniThreadLibrary, go to www.omnithreadlibrary.com.

Forward record declaration

Forward declaration is not really a new concept. It was already present in the original Wirth Pascal where it allowed the programmer to do one thing and one thing only – call procedure A from procedure B and procedure B from procedure A. Remember – in ye olde times of good old Pascal we had no interfaces, no classes, no units … just procedures and functions.

Because the code tells more than thousand words, here’s an example (still perfectly valid in modern Object Pascal).

procedure ProcA; forward;

procedure ProcB;
begin
  ProcA;
end;

procedure ProcA;
begin
  ProcB;
end;

OmniThreadLibrary 3.07

New year, new release ;) You can get it as a zipped download, git checkout, via Delphinus, and via GetIt.

For more information on OmniThreadLibrary, go to www.omnithreadlibrary.com.

OmniThreadLibrary 3.07 beta + plans for OmniThreadLibrary 4.0

It is almost time for a new release! If you’d like to see what 3.07 will contain – or if you just want to test it before it is out – click here.

[If you are a serious OTL user - and especially if you are using pre-Seattle Delphi - please test your program(s) with the beta release. Thanks!]

I have started work on OTL 4.0 which will (fingers crossed) fully support cross-platorm work. Yup, it will work on all platforms that Delphi can compile for! Multiplatform support was implemented by Sean B. Durkin so if OTL/Mobile is something you were waiting for a long time, go to OmniThreadLibrary-For-Mobile Google+ community and express your gratitude.

RAD Studio and Version Control + Git – links

This post contains just a bunch of links that I refered to in my two-part (live) presentation on version control management with RAD Studio and Git.

Slides for the presentation: pps, pdf

Embarcadero Akademija: RAD Studio and Version Control

This is an announcement for the next Slovenian “Embarcadero Academy” session. As it is intended for Slovenian Delphi programmers, the rest of this post is in the Slovenian language.

Debug Faster. Debug Smarter.

My CodeRage XI session on debugging in RAD Studio is now available on YouTube.

If you have missed the presentation or want to see it again, now is a good time :)

Additional materials (slides, code) are available on the Presentations page.

Hear me speak at CodeRage XI

This year I’ll be presenting two sessions at CodeRage XI.

On Wednesday, I’ll be exploring the topic of debugging in RAD Studio. I’ll show how advanced options in breakpoint properties work and drop some debugging tips & tricks here and there. Join me in room 1 at 7 am PST / 10 am EST / 16.00 CET.

On Thursday I’ll be demoing new enhancements in FastMM. If you are writing multithreaded code, then this session will be targeted to you! Join me in room 1 at 10 am PST / 1 pm EST / 19.00 CET.