Chapter 12
LINQ
LINQ had been introduced with Visual Studio 2008 and .NET 3.5.1. One of the main creator of LINQ was no other than computer programming languages legend Anders Hejlsberg, inventor of Turbo Pascal, Delphi and one of the architects of C# and later of TypeScript_too. When _Anders came to Microsoft in 1996, his first task was to port an existing Java UI library to Windows which would become later WinForms within the .Net Framework.
The idea behind LINQ
The revolutionary idea of LINQ was to abolish the "impedance" between code and data. That means that developers will be able to fluently combine code with data for queries against any kind of list without having to resort to any kind of method calls.
Take for example the filtering of a list of strings.
Without LINQ it would require the call of a filter() method.
var cities = String[]{"Esslingen", "Plochingen", "Stuttgart"}
var smallCities = filter(cities, "*.gen")
With LINQ the filtering can be combined with a list in a more readable statement:
var cities := List<String>{}{"Esslingen", "Plochingen", "Stuttgart"}
var smallCities := From City In Cities Where City.EndsWith("gen") Select City
This time the filtering is part of the syntax and not done by function. What is more important is that filtering is only one of many options. LINQ queries can be long.
Although LINQ is about querying lists and arrays, updating elements of a list or creating new elements is possible as part of the query.
LINQ and SQL
Although the syntax in the second line looks like SQL, LINQ has nothing to do with SQL (there had been already endless debates about the fact that a LINQ query starts with From and not with Select for example).
It's important to note that the result of a query is not an array or a list. It's a special System.Linq.Enumerable+WhereListIterator`1 object that can be enumerated like a typed list. The fact that any LINQ query is just the definition of the query and not the result is called lazy evaluation.
And City is just the name of a local variable that derived its type from the fact that Cities is a List
Example 12.1: (XS_LinqExample1.prg) ** You will find the source code in the repository
The LINQ syntax is not the only option of course. Since the "magic keywords" like Where and Select are based on extension methods, they can be called directly of course.
**Example 12.2: (XS_LinqExample2.prg) **
// Using LINQ with extension methods
using System.Collections.Generic
using System.Linq
Function Start() As Void
var cities := List<String>{}{"Esslingen", "Plochingen", "Stuttgart"}
var smallCities := cities:Where({city => city.EndsWith("gen")}):Select({city => city})
ForEach var city in smallCities
? city
Next
Which one is better? Although it is a matter of taste, of course, I would prefer the second option because it follows more the traditional thinking of calling methods that return something (even its the same iterator object).
The main advantage of the Where extension method (and LINQ in general) over functions like AScan is that the methods always return typed objects. That's the reason why Visual Studio offers IntelliSense when working with the returned objects.
Don't forget to include the System.Linq namespace. Otherwise, the where extension method cannot be found by the compiler.
LINQ and databases
What about databases? Wasn't LINQ supposed to be some kind of "object layer" above the database API? As already stated, LINQ itself has nothing to do with databases.
There used to be a "LINQ for SQL" when LINQ came out in 2008 but Microsoft axed the project in favor of the Entity Framework which is an ORM (Object Relational Mapper) outside the .Net runtime. The Entity Framework is the official ORM from Microsoft and is an integral part (or maybe better an addition) of .Net 7 and beyond.
But of course, LINQ can the used to query datatables and their rows. But since each field in a row is not typed the query loses a little bit of its simplicity.
For the next example imagine a tiny DataTable that contains a few (data) columns. A query is needed that returns all columns with a name that starts with an 'A'.
**Example 12.3: (XS_DataTableLinq.prg) **
// Example for selecting rows with LINQ
using System.Data
using System.Linq
Function Start() As Void
var ta := DataTable{"Cars"}
ta:Columns:Add(DataColumn{"Id", typeof(Int)})
ta:Columns:Add(DataColumn{"Name", typeof(String)})
ta:Columns:Add(DataColumn{"Year", typeof(Int)})
ta:Columns:Add(DataColumn{"Country", typeof(String)})
ta:Columns:Add(DataColumn{"Price", typeof(Double)})
var r := ta:NewRow()
r["Id"] := 1
r["Name"] := "De Tomaso P72"
r["Year"] := 2019
r["Price"] := 1.3E6
r["Country"] := "Italy"
ta:Rows:Add(r)
r := ta:NewRow()
r["Id"] := 2
r["Name"] := "La Ferrari"
r["Year"] := 2013
r["Price"] := 1.4E6
r["Country"] := "Italy"
ta:Rows:Add(r)
r := ta:NewRow()
r["Id"] := 3
r["Name"] := "Aston Martin Vulcan"
r["Year"] := 2014
r["Price"] := 2.3E6
r["Country"] := "England"
ta:Rows:Add(r)
r := ta:NewRow()
r["Id"] := 4
r["Name"] := "Koenigsegg Gemera"
r["Year"] := 2020
r["Price"] := 2.3E6
r["Country"] := "Sweden"
ta:Rows:Add(r)
// var rows := from row as DataRow in ta:rows where row["Country"]:ToString() == "Sweden" select row
var rows := from row in ta:rows:Cast<DataRow>() where row["Country"]:ToString() == "Sweden" select row
ForEach row As DataRow in rows
? i"The {row[""Name""]} is from {row[""Country""]}"
Next
rows := from row as DataRow in ta:Rows where row["Country"]:ToString() =="Italy" && Double.Parse(row["Price"]:ToString()) > 1300000 select row
ForEach row As DataRow in rows
? i"The {row[""Name""]} costs {row[""Price""]}"
Next
Since all DataColumns are part of the System.Data.DataColumnCollection for which the where extension method does not apply, the compiler cannot determine the type of d so that in X# the variable has to be declared explicitly.
Not a big deal of course. Note that the columns will be returned sorted by their column name. That's cool.
LINQ and WinForms
Although there is of course no official connection between LINQ and WinForms, WinForms controls are just another kind of object, LINQ queries can be really helpful for selecting controls based on their type or any other attribute.
The following example lists all controls that are child controls of a GroupBox control based on their type.
The query, that is part of the example, is hopefully not very difficult to understand:
(FROM c AS Control IN ((GroupBox)container ):Controls:OfType<Label>() SELECT (Label)c):ToList()
This LINQ query returns all label controls that are child controls of a GroupBox as a generic List - but of what type?
Example 12.4: (XS_WinFormsContainerControlsLINQ.prg) ** You will find the source code in the repository
Practicing LINQ with LINQPad
For anybody who likes to "dive deeper" into LINQ and all the possibilities that come with it, LINQPad by Joe Albahari is a kind of must-have tool because it makes working with LINQ queries simple and rewarding.
The main advantage of LINQPad is querying the database over a connection that had been set up first.
LINQPad is free but there is also a "premium version" which I recommend (at least to support the developer of this really helpful tool)
LINQPad can process statements and expressions in C#, F#, and VB but not in X# (maybe someone will develop an extension in the future). This is not a disadvantage since the X# development stayed as close as possible to the C# syntax.
Last but not least: There are some really good LINQ examples in the X# help file. The current version of LINQPad is 7 but the tool was a great assistant from the first version on.
Fig 12.1: LINQPad makes trying out LINQ queries both easy and convenient and is a great learning tool
A (very short) look behind the scenes
LINQ is based on a few (simple) ingredients provided by the X# compiler:
- Type inference for local variables
- Extension Methods
- Lambdas
- Anonymous types
- Query comprehensions
Type inference for local variables
The compiler can interfere with the type of a local variable through the assignment so there is no need for a type (that's how var works)
Extension Methods
Any method can attached to a class without having to extend the class definition itself. A very handy feature.
Lambdas
A function is just the body - there is no need for a formal declaration.
Anonymous types
A type can be created without a formal type definition like a class definition.
Query comprehensions
Statements like From, Where, Select, etc can be used as part of the source code without being a formal part of the language definition itself.
There are nearly 40 extension methods mostly for Enumerable
Lazy evaluation
One key feature of LINQ is called lazy evaluation. That means that a LINQ query defines only the query, the expression tree. The compiler translates the query into an expression tree. Only when the query is used inside a loop or a ToList() method is appended, the query will be run and therefore the CPU burns cycles.
| Method | What does it? |
|---|---|
| Select | Selects the "output" object |
| Where | Filters the objects based on a lambda expression |
| SelectMany | Select the output object from multiple clauses |
| OrderBy | Sorts the objects |
| Cast | A type conversion with each object |
| GroupBy | Groups objects |
| Join | joins two or more lists |
Tab 12.1: A few of the "LINQ methods" that are extension methods of IEnumerable
LINQ and debugging
One of the early criticisms of LINQ was the minimal debugging support in Visual Studio. Even a complex query could only be debugged in a single step. This has improved a lot since the first version. In recent versions of Visual Studio parts of a LINQ statement can be debugged by pressing [F11] which makes debugging (and understanding) a lot easier.
LINQ examples
The best way to explain LINQ and show its potential for the simplification of common techniques like filtering, sorting or grouping is by showing small examples.
Iterating through all elements of a list but the last one
X# is not Python (the language) so there is no slicing syntax which makes accessing elements of a list very simple (and not every part of the C# syntax is already part of X#). So iterating over all elements of a list that is provided by the method would mean an extra step of assigning the elements to a variable, using the index notation and iterating over the variable. With LINQ it becomes a little easier or different, depending on your point of view.
**Example 12.5: (XS_IteratingListExeptLastElement.prg) **
// Iterating over a list except the last element
using System.Collections.Generic
using System.Linq
Class C
Static Method GetNumbers() As List<Int>
Return List<Int>{}{11,22,33,44,55,66}
End Class
Function Start() As Void
ForEach var z in C:GetNumbers():ToArray():Reverse():Skip(1):Reverse():ToList()
? z
Next
Now for some readers, the following statement might seem a little like a joke:
ForEach var z in C:GetNumbers():ToArray():Reverse():Skip(1):Reverse():ToList()
Why do I have to chain five (!) methods together just to achieve something that could be done with a simple loop? And what about the performance if this would be a very large list? I know that many experienced developers will have a problem with this approach and I am with you. The reason for having to turn the List
So this example is more like an example of the flexibility that comes with LINQ if needed. And it's always good to have options.
With .Net Core there is a SkipLast() method and I am sure that with X# 3.0 (or beyond) X# will have the range operator like in C#.
Converting a CSV file into objects
A CSV file is a "Comma Separated Values" text file although the separator does not have to be a comma and could be any valid character. CSV is still a very popular text file format for exporting data because of its simplicity.
The following examples show how easy it is to convert a CSV file line by line into objects of a certain class with a LINQ query. The text file is a file named 'Albumtitles.txt' that consists of several rows:
Title,Interpret,Year
Boys And Girls,Bryan Ferry,1985
Nightshift,Commodores,1985
Brothers in Arms,Dire Straits,1985
**Example 12.6: (XS_ConvertText2Objects.prg) **
// Using LINQ to convert a Csv file into objects
Using System.IO
Using System.Linq
Class Album
Property Title As String Auto
Property Interpret As String Auto
Property Year As Int Auto
Override Method ToString() As String
Return i"Title: {Self:Title} Interpret: {Self:Interpret} Year: {Self:Year}"
End Class
Function Start() As Void
var txtPath := Path.Combine(Environment.CurrentDirectory, "Albumtitles.txt")
// really "cool" how easy it is to skip first line with the header names
var albums := from line in File.ReadAllLines(txtPath):Skip(1) Select Album{}{title:=line:split(c",")[1],;
interpret:=line:split(c",")[2],;
year:=Int32.Parse(line:split(c",")[3])}
ForEach var album in albums
? album
Next
TIP: This is a very nice website for playing around with CSV data in a browser where everything stays in the browser and will not send over the Internet: https://whattheduck.incentius.com. The website uses DuckDB which allows SQL queries on any CSV file in the browser. It has nothing (or little) to do with X# though, but it proves again there is always something new that nobody has invented before.
Selection Controls based on their types
Of course, there is no "LINQ for WinForms" in the .Net runtime. But if you already understand the idea of LINQ, you know that such a direct connection is not necessary at all. LINQ queries work with any Array or List (with a collection it depends on the implemented interfaces).
What about querying WinForms controls, that are part of a container, based on their type? Like getting only labels? Or buttons with specific property values? Using a LINQ query is more elegant, simpler, and in the end better readable than a traditional loop.
The following query returns all controls that are child controls of a GroupBox and are Labels:
(FROM c AS Control IN ((GroupBox)container ):Controls:OfType<Label>() SELECT (Label)c):ToList()
Would a Where() method be simpler and better readable? In theory. It may be surprising at first, that the following query does not work:
(((GroupBox)container):Controls):Where({c => c IS Label}):ToList()
The compiler error is clear: A ControlCollection has no Where method. It took me a few seconds to search StackOverflow to know what the reason was (I should have known anyway). The ControlCollection does not implement IEnumerable
((GroupBox)container):Controls:Cast<Control>():Where({ c => c IS Label}):ToList()
Maybe at this point, you are just happy that you have 100% confidence in the fact, that a traditional loop and an if statement always work. But, I can only encourage you to give LINQ another try.
Ex
Advanced LINQ
There is more to LINQ than a simplified querying syntax for arrays and lists and extension methods for IEnumerable
And finally: create your expression trees for more flexibility
By now it should be clear that LINQ is both a very flexible and developer-friendly extension for dealing with any kind of lists and arrays. But all the examples had one thing a common, they used a fixed expression. In some scenarios it would be more flexible with the expression could be constructed dynamically. This is possible too by creating a custom expression tree that is compiled into a LINQ expression that can be used the same way as any other LINQ expression. While this may sound complicated, in the end, it isn't. As usual, there are so many examples available that be used both for learning or taken 1:1 for a specific requirement.
Some examples will follow in one of the future releases of this chapter.