Planet Squeak

blogs about Squeak, Pharo, Croquet and family
planet squeak - planet squeak es - planet squeak jp - planet croquet - planet squeak code - planet smalltalk

February 18, 2020

PharoWeekly

[Ann] new version of concurrent Prog booklet

Thanks for all the feedback. I fixed the embarassing english mistakes and french left over. I just issued a new version of the booklet.

http://books.pharo.org/booklet-ConcurrentProgramming/

S.

by Stéphane Ducasse at February 18, 2020 01:03 PM

February 14, 2020

PharoWeekly

[Ann] Teachable

I moved Teachable project now to GitHub:

https://github.com/astares/Pharo-Teachable

The Project shows the power of Smalltalk meachanisms within Pharo and
can be useful for dynamic mock object building in unit tests.

Docu and load instructions are included in the project site.

Side note:
==========
You can also easily load it via catalog within Pharo 8/9:

– take a fresh clean Pharo image using PharoLauncher
– click on “regular network connection” in the Quick setup sectiono of Pharo welcome window
(this allows spotter to show catalog projects)
– open Spotter (SHIFT+ENTER) and key in “Teachable”, hit ENTER to load via catalog

Another possibility is to search for “Teachable” in Catalog utility directly (Tools -> Catalog Browser).

Have fun
T.

by Stéphane Ducasse at February 14, 2020 06:23 PM

February 13, 2020

Torsten Bergmann

Pharo-VistaCursors

Pharo-VistaCursors is a older project I wrote in the early days of Pharo to have a better cursor on Windows.

I moved it over from SmalltalkHub to GitHub.

Bild

 It is easy to load via Pharo catalog or load instructions from https://github.com/astares/Pharo-VistaCursors

by Torsten (noreply@blogger.com) at February 13, 2020 09:24 PM

Casino - Java to Angular migration using Pharo

Casino: A Pharo smalltalk project that migrates web projects from Java to Angular

https://github.com/badetitou/Casino

by Torsten (noreply@blogger.com) at February 13, 2020 09:18 PM

February 12, 2020

PharoWeekly

[Ann] uFFI booklet

Hi all,

We have been working a lot in the last months in the documentation of uFFI, and we are happy to announce a first stable version of it that you’ll find in a link below 🙂
The booklet talks about uFFI, how to use it, how marshalling works, how to use structures, unions, etc.
And it is aimed for the version present on Pharo8.0.
https://github.com/SquareBracketAssociates/Booklet-uFFI/releases/tag/v1.0.1
Of course, a v1 does not mean it’s perfect. Future versions will include new chapters/features that are on the pipeline such as callbacks and how to specify string encodings in your bindings.
Also, so if you find something to fix, to add, to remove, or something hard to follow, let us know
 – by email or
 – by issue (will be easier to track actually than an email) or
 – by PR (and we will have a boost of happiness in addition)
Cheers,
Guille in behalf of everybody that helped and contributed to this

by Stéphane Ducasse at February 12, 2020 05:02 PM

February 11, 2020

PharoWeekly

About Bloc design

https://medium.com/feenk/one-rendering-tree-918eae49bcff

by Stéphane Ducasse at February 11, 2020 10:20 PM

February 10, 2020

PharoWeekly

[Ann] Concurrent Programming in Pharo

Is now available on http://books.pharo.org/booklet-ConcurrentProgramming/

S.

by Stéphane Ducasse at February 10, 2020 11:58 AM

February 09, 2020

PharoWeekly

About the Pharo ecosystem

Hi,

Let’s take a moment to contemplate on the breath and the depth of the expanding Pharo ecosystem. So much is happening that it is very hard to keep track, let alone look at everything or try it out.

Marcus’ excellent curated Pharo Newsletter is one place to see this.

Take the February 2020 list of New/Updated Libraries and Frameworks at the end:

https://mailchi.mp/pharo/pharo-newsletter-february-2020

This is one of the longest enumerations I have seen so far. The great thing is that it is like that almost every month. Check out the archive:

https://us11.campaign-archive.com/home/?u=6f667565c2569234585a7be77&id=048680a940

Of course, GitHub is also a good way to see this happening. The main entry being:

https://github.com/pharo-project/

Which contains several overviews:

https://github.com/pharo-project/PharoMap

https://github.com/pharo-open-documentation/awesome-pharo

Topic tags automatically organise some projects:

https://github.com/topics/pharo

https://github.com/topics/pharo-smalltalk

What all this comes down to is that Pharo has many recently developed, actively maintained options to get your job done: to model your domain, to build your user interface, to talk to other systems, to speak other protocols, to interface with the world, to deploy and to deliver your applications.

Similarly, Pavel recently did a very good job at the describing why Pharo itself is so great:

https://github.com/pavel-krivanek/pharoMaterials/blob/master/features/PharoKeyFeatures.md

So thanks to all of you for helping to make Pharo into what it is.

Thanks to our users: for your questions, you’re feedback, your bug reports.

Thanks to those helping out others on the mailing lists.

Thanks to everyone who ever blogged or otherwise wrote about their experiences with Pharo.

Thanks to the contributors involved in constantly improving Pharo with Pull Requests, to those working hard to maintain the process and the machinery behind the development process.

Thanks to all the developers producing and maintaining the many libraries and frameworks that help us in our day to day work.

Thanks to those writing documentation.

Thank you.

Sven

PS: This is already a long email, but I know that I forgot many important points, feel free to add them in reply.


Sven Van Caekenberghe
Proudly supporting Pharo
http://pharo.org
http://association.pharo.org
http://consortium.pharo.org

by Stéphane Ducasse at February 09, 2020 11:24 AM

February 08, 2020

Program in Objects

The Results Are In!

Round 1 of The James Robertson Memorial Programming Competition (JRMPC) just completed. Congratulations to Team ‘The Battle of Waterloo’ from Woodbridge College in Toronto, Canada for an outstanding performance.

We look forward to Rounds 2, 3, 4, and 5 (the prize-winning round). Let’s give a hand to these exceptional students and newfound Smalltalkers!

by smalltalkrenaissance at February 08, 2020 08:30 PM

February 06, 2020

PharoWeekly

[Ann] PolyMath 1.0.2

Dear all,

we are happy to announce the release of PolyMath 1.0.2, the
computational framework for Pharo. This release works on Pharo 7.0 and Pharo 8.0. All the 815 tests and the CI are green on MacOs, Windows and Linux 🙂

Please find more information about the project here:
https://github.com/PolyMathOrg/PolyMath

Changelog of modifications since version 1.0.1 is available here:

https://github.com/PolyMathOrg/PolyMath/milestone/4?closed=1

You can install the 1.0.2 version in a fresh Pharo 7.0&8.0 with the
following code snippet:

Metacello new
repository: ‘github://PolyMathOrg/PolyMath:v1.0.2/src‘;
baseline: ‘PolyMath’;
load

You can join the dev team on #polymath channel for weekly meetings on Thursday. We are looking for more people for tests and contributions.
I’m starting a small experiment. If you want to support me working on PolyMath or others of my project like Tensorflow Pharo bindings, you can be one of my github sponsors.
Please give a monthly support here:

by Stéphane Ducasse at February 06, 2020 01:47 PM

February 04, 2020

PharoWeekly

[Ann] String Slices

Hi,
I was working on parsing longer texts, and my program was extracting substrings which sometimes were rather large. Not that it actually caused any problems, but it kind of hurt my feel for good code to do that.
It has most surely been done before, but here it is again then, string slices.
https://github.com/kasperosterbye/StringUtilities
A string slice a substring obtained by a start and an end index into an existing string. It is thus possible to get substrings without copying the bytes of the original string – at least for a large number of operations.
I have spend a lot of energy in getting it to work right with both ByteString and WideString, and to let string slices to be protocol compliant with the whole String protocol (except slices are read-only).
Feel free to enjoy and laugh.
In the work with doing StringSlice as a subclass of String, I collected a few issues in String. Those are mentioned at:
https://github.com/kasperosterbye/StringUtilities/blob/master/StringIssues.md.
The baseline does not load the tests, but they are there if you want to check that too.
I have tried the library on Pharo 7, 8 and 9 (as of 2020-02-04).
Best,
Kasper

by Stéphane Ducasse at February 04, 2020 09:40 PM

Hernan Morales

nCoV-2019 basic analysis with Pharo

I pushed two Pharo scripts which may help someone to do a basic analysis of the novel virus: One script is a one-liner to download the reference genome form NCBI and the other one downloads the publicly available sequenced genomes, reports some basic statistics, filter out genomes which are not complete, and align the sequences with MAFFT. You can access the repository code here: https://github.com/hernanmd/nCoV-2019

by Unknown (noreply@blogger.com) at February 04, 2020 06:51 PM

Program in Objects

Fuzzy Thinking in Smalltalk

by Lorenzo Schiavina
EDOR Metodi Quantitativi

When I was a professor of Operation Research at the Faculty of Mathematics at the Catholic University of Brescia, I was lucky enough to contact Lofti Zadeh who was going to develop fuzzy logic and I was impressed by his work.

Subsequently I deepened the topic by reading the excellent book by Bart Kosko, Fuzzy Thinking, and I began to get interested into the approach and to convince myself that this technology could certainly be an extension of the models that I was used to implement.

As I have been dealing with computer science for more than 20 years and I was familiar with all the languages ​​and computers of the time and I was lucky enough to be the first in Italy to know Smalltalk, I presented to the ESUG (European Smalltalk User Group), held in Brescia at my University, plans for an extension of Smalltalk classes (FuzzyWorld) that would be able to deal with fuzzy logic.

The first applications that I tried to develop convinced me that the tool offered absolutely unique possibilities and therefore I deepened my research thanks to two exceptional books.

The two books are:

I was fortunate to develop an application for a premier league athletic soccer trainer and the result was excellent: for 3 years in a row, this trainer (moreover of non-top teams) was among those who had the least injuries; his testimony on my contribution was fundamental.

Since then, I have made fuzzy applications in the most diverse fields, integrating fuzzy logic with optimizations through genetic algorithms.

Fuzzy logic is a tool that has been proven useful in dealing with and solving very complex problems; among these, forecasting problems are certainly to be included.

Since in the beginning of my activity I have had the opportunity to deal with investment problems, it seemed very interesting to do an experiment using this type of approach; the application was named FuzzyStock.

I therefore identified (in a completely random way) a stock listed on the stock exchange to verify the results that the use of this (discussed) computer technology could offer me.


The following pages show the data collected and their characteristics:


Synthesis window of the S&P title data used for the experiment

The input information of the experimentation is reported in the 3 areas:

Title (S&P)

o Number of available data (682)

o Date of training surveys (02/01/01 to 24/09/03)

o Size of training data (598)

Processing specifications

This is the fundamental point of the model: the training algorithm (a moving average implemented in fuzzy logic) which was the heart of the system, was applied to data from 1 to 598; the block (i.e. the fundamental training unit) was 9 data, starting with the first available data.

From the experience of analyzing the structure of the first 598 data, the algorithm had to “learn” how to move to evaluate the data from 599 to 682 and demonstrate whether it had “understood” or not how to make predictions; the time interval of the forecast ranged from 28/05/03 to 24/09/03 and the knowledge deriving from the analysis of data from 1 to 598 was applied to this interval

Strategy specifications (demonstration not included)

The strategy specifications identified the elements for simulating the model’s performance (that was not fuzzy oriented):

Buying filter: percentage of growth in the price of the security to make a purchase

Selling filter: percentage of decrease in the price of the security to make a sale

Stop loss filter: percentage of error to decide the abandonment of the chosen strategy (purchase or sale)

Void selling filter: percentage of variation for short selling

Wrong forecasting filter: error percentage for changing strategy (purchase or sale)


The execution of training time of the model is about 6 seconds, after which the result of the forecast is presented:

The result shows both the value of the fuzzy model and the “traditional” one of the trend (the old approach I used), the model of which is shown on the third line of the display; on the fourth, it is possible to indicate a percentage uncertainty assessment to be applied to the reported price that can be entered by a “human” expert.

The response columns are as follows:

  1. Detection date
  2. Lower value of the acceptance interval of the day chosen by the user through the value of probability (90% in the model)
  3. The prediction made by the model
  4. Upper value of the acceptance interval of the day chosen by the user; obviously the value of column 2 and column 4 identify the acceptable price or not; outside the range, the forecast is incorrect
  5. Real price of the day
  6. Forecast error due to exit from the acceptance interval
  7. Forecast provided by the use of the trend model shown on the third row above

On line 6 you can see the indication of out range error (forecast 973.27; real minimum 971.42 i.e. an over estimation).

The data can be stored for subsequent evaluations, brought to Excel or shown.

Using the Show trend button, it is possible to view and graphically analyze the result of the forecast (both fuzzy and trend numerical value in the lower part, where the trend parameters are evaluated):

At the first line the number of last period observations; in red the price of the day; green forecasts for the trend; fuzzy ones in blue

The result of the fuzzy model is particularly interesting (obtainable by disabling the Trend flag) and using a larger series, in this case 50 items):

By eye it can be seen that the maximum forecast error occurred between 20/08 and 7/09; by clicking on the point, you get the detail of the observation:

As you can see, the maximum error (Scarto percentuale) made by the fuzzy approach has a difference of less than 2% of the price.

Pleased, notice that this error was the biggest in the series, excluding a specific case I am going to point out .

It is interesting to note that a terrorist attack occurred in Madrid during the trial period.

Obviously, the result of this event has profoundly influenced the stock market price (and I suppose this was the reason for wrong forecasting).

Of course this event is known to operator, so the processing window had been modified adding the possibility for the human operator to insert his personal evaluation to the forecast, obviously dependent on the external events observed.

by smalltalkrenaissance at February 04, 2020 12:08 PM

January 31, 2020

PharoWeekly

[Ann] Pharo map

There are many pharo- organizations and google does not find them easily.

Now there is PharoMap to get a root of the pharo-organisations.

https://github.com/pharo-project/PharoMap

S.

by Stéphane Ducasse at January 31, 2020 10:38 AM

January 30, 2020

PharoWeekly

[Ann] Pharo 4 Pharo mailing-list

Hi

Tired of boring and dull discussions about the past?
Tired and bored about the fact that the discussions are not about Pharo but about X, Z, K?
Want to get in touch with other real pharo users?
Want to have positive and nice discussions about Pharo?
This mailing-list is for you
https://lists.gforge.inria.fr/mailman/listinfo/lse-pharo4pharo
I created a new mailing-list for Pharo 4 Pharo.
The idea of this new mailing-list is to exchange between Pharo users on Pharo.
If you want to talk about the weather and other cool fishing practices or how other languages are super cool,
then you should use another mailing-list.
You can consider that I’m
[ ]  an asshole
[ ]  an arrogant idiot
[ ]  other
This is not my concern.
My concern is: can we respect Pharo and users of Pharo that want to share what they are doing
and ask questions about their language?
If you join this mailing-list be warned that I (yes me the benevolent dictator of Pharo) will remove you from the mailing-list
when you get on my nerves.
If you do not like it, do not like me, do not appreciate it, do not join!
S.
BTW they are plenty of newsgroups about Smalltalk and you can join there.

by Stéphane Ducasse at January 30, 2020 07:34 AM

January 29, 2020

PharoWeekly

[Ann] A new version of the headless VM

A new version of the headless VM is available.

It can be downloaded from:

https://files.pharo.org/vm/pharo-spur64-headless/win/PharoVM-8.3.0-b612fd5f-win64-bin.zip
https://files.pharo.org/vm/pharo-spur64-headless/linux/PharoVM-8.3.0-b612fd5-linux64-bin.zip
https://files.pharo.org/vm/pharo-spur64-headless/mac/PharoVM-8.3.0-b612fd5-mac64-bin.zip

Or more easily using ZeroConf:

With the image
$ wget -O – get.pharo.org/64/90+vmHeadlessLatest | bash

Without
$ wget -O – get.pharo.org/64/vmHeadlessLatest90 | bash

This new version has a series of bugfixes and the following features:

– Update TFFI to v1.2.0: Allowing better marshaling and callbacks from
outside threads.
– Update README.md
– Add instructions on how to create a vmmaker image.
– Fixing UnixOSProcessPlugin
– Redefinition of squeakFileOffset
– Generating include files as an artifact
– Adding a configurable strategy for reading / writing the image
– Building using Musl Libc
– A cleaner implementation of the print to stdout and file.
– OSX File Dialog
– OSX icon customization
– OSX customization for apps.
– Adding build on GitHub actions

I will like to thank all the contributors specially Guille, Esteban,
Ronnie. And also, Feenk and Schmidt that they are using it, reporting
issues and contributing.

Just a friendly reminder, if you want to contribute, you are always welcome!!

https://github.com/pharo-project/opensmalltalk-vm

Cheers,
Pablo


Pablo Tesone.
tesonep@gmail.com

by Stéphane Ducasse at January 29, 2020 05:15 PM

Program in Objects

Extending the Smalltalk Syntax 4

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 4: Hybrid Compilation

Have you ever heard of the idea the creators of Smalltalk had for allowing any class to choose its compiler? To provide support for this classes respond to the #compiler message before the actual compilation is attempted. Why then, this capability hasn’t been exploited yet? Are you willing to investigate it further? OK, bear with me.

In Story 3 of this series we discussed how to inline TaggedNodes in a Smalltalk method. We mentioned several applications of this capability and took JSON as a basic example. Today we can take a similar approach and try to see how to do our exploration with JSON in mind. From there it will become fairly clear how to proceed in other cases. So, let’s put ourselves this objective: compile the following method in our Smalltalk dialect:

jsonCoordinates
  <json>
  ^{
      "latitude": 48.858093,
      "longitude": 2.294694
    }

Where to start? Here is the roadmap:

  1. Discuss the introduction of pragmas for enabling foreign compilation.
  2. Introduce the HybridCompiler class.
  3. Generate the hybrid method when there are no arguments.
  4. Introduce the ParametricString class.
  5. Generate the hybrid method when there are arguments.

Task 1: Discussion


Let’s start by noticing how our example above is slightly different from what we did in Story 3. Here <json> is not a tag but a pragma (there is no closing tag). We are using this pragma to make it clear that we will be using a foreign compiler.

A similar example with JavaScript

canSimulateCase
    <js>
    $scope.canSimulateCase = function(c) {
        return !$scope.isProcessingCase(c)
    };

One difference with tagged nodes is that here the entire method body is written in a foreign language. Why this variation is interesting? Because it will allow us to pass Smalltalk arguments to foreign methods. Like this:

jsonCoordinates: lat longitue: long
  <json>
  ^{
      "latitude": #lat,
      "longitude": #long
    }

meaning that the foreign source code will be generated dynamically.

Note that I’ve used # to mark what follows as an argument. We don’t want to replace every occurrence of 'lat' and 'long' with the arguments; do we?, so we need to tell where we want the replacements to happen. The reason for using # as a marker is that it presents (almost) no collision with foreign tokens.

Task 2: Hybrid Compiler


If we get back to our examples above, we will see that these methods have two parts: (1) a Smalltalk header including the pragma and (2) the foreign code. This accounts for hybrid compilation. We need to, at least, parse the beginning of the method to read the pragma that tells which compiler to pick, and then pass it the body. For doing all of this we will need the following class

Object
    subclass: #HybridCompiler
    instanceVariableNames: 'source smalltalk foreing method'
    classVariableNames: ''
    poolDictionaries: ''

The smalltalk ivar is initialized to the Smalltalk compiler, and foreign to the parser (or compiler) associated to the method’s pragma. When the source is set, the smalltalk compiler is used to read the pragma ('json' in our example). At this point the Registry (see Story 3) will provide us with the foreign parser. If there is no pragma or there is one which is not in the Registry, the compilation is entirely on smalltalk. The method ivar will hold the compilation result.

Task 3: Hybrid method


Once an instance of HybridCompiler has been initialized, it is time to compile the method. For now we will assume that there are no arguments (unary case).

HybriCompiler >> compile
  | cm |
  foreign isNil ifTrue: [^method := smalltalk compileMethod: source].
  ast := foreign parse: self body.
  cm := smalltalk compileMethod: self template.
  method := ForeignMethod from: cm.
  method
    sourceCode: source;
    foreignCode: ast format;
    foreignParser: foreign

There are several things to explain here:

In the unary case we are now, the #template method has the following source code

template
  ^self selector , '
  #code.
  #parser.
  ^#code'

where #selector answers, with the help of the smalltalk compiler, the method’s selector; the following two symbols are placeholders for two slots in the literal frame that we will change below. Note that the compiled method will answer with the contents of the first literal slot.

ForeignMethod >> foreignCode: aString
  self literalAt: 1 put: aString
ForeignMethod >> foreignParser: aParser
  self literalAt: 2 put: aParser

The second literal holds the foreign parser, which may be needed for further processing (e.g., painting the source code).

Task 4: Macro Expansion


We will now address the case where the method has arguments which are inserted in the foreign code using # as the marker prefix. Let’s refer to these prefixed identifiers as hashed tokens.

What we need is to dynamically replace all hashed tokens with the corresponding arguments. For doing this we will introduce a new class named ParametricString, which will implement this transformation as a service.

Basically this new object takes aString including hashed tokens and a sequence of all possible tokens (in our case, the sequence of method arguments). Using this information the object produces a sequence of strings and indexes. The strings are the fragments between hashed tokens, the indexes refer to the sequence of hashed tokens. For instance if the inputs are:

the object should generate the following sequence:

#('hello ' 3 ', this is a ' 1 '.')

with the somewhat clearer Squeak-braces syntax, this would be

{'hello'. 3. ', this is a '. 1. '.'}

Later on, when the object is required to expand the tokens using actual arguments it will replace the indexes with the corresponding values, concatenating them all. The message to do this will be

aParametricString expandWith: arg1 with: arg2 ...

Task 5: Hybrid method with arguments


Since we have already worked on the unary case in Task 3, we only need to redo the #template method for the case where there are arguments.

The first change to consider is that what before was simply the selector, it is now a keyword message with the formal arguments. This can be simply accomplished with the help of the smalltalk compiler, so I will not go in detail here. We will just assume that self selector will answer with the signature of the method (i.e., 'keyword1: arg1 keyword2: arg2'…).

Since the source code will be now a bit more complex, I will use the #streamContents: constructor.

template
  | arguments processor |
  arguments := self arguments.
  processor := ParametricString from: self body tokens: arguments.
  ^String streamContents: [:strm |
    strm
      nextPutAll: self selector;
      crtab;
      nextPutAll: '| ast |';
      crtab;
      nextPutAll: '#code.';
      crtab;
      nextPutAll: '#parser.';
      crtab;
      nextPutAll: 'ast := #parser parse: (#code expandWith: '.
    arguments
      do: [:arg | strm nextPutAll: arg]
      separatedBy: [strm nextPutAll: ' with: '].
    strm
      nextPutAll: ').';
      crtab;
      nextPutAll: '^ast format']

A key remark here is that in order to connect processor with: #code we need to make sure we plug the processor in the first literal slot, where #code acts as its placeholder. This is achieved by a modification to the #compile method we saw in Task 3. Instead of sending ast format as the argument of foreignCode: we need to send

ParametricString from: code tokens: self arguments

Final words


As always, the code presented here is just a sketch of the actual code, which might end up being a bit more complex (but not too much!). Also, there are lots of details in my description for which I haven’t included any code at all. The reason is two-fold. Firstly, my goal is to provide a roadmap rather loadable code. Secondly, I don’t want to capture all the fun just for myself.

by smalltalkrenaissance at January 29, 2020 01:29 PM

Rocking Smalltalk

Another fantastic video from our Smalltalk devotee:

I didn’t want to be left out, so I created my own video:

graphics live coding with Athens and Pharo

by smalltalkrenaissance at January 29, 2020 01:24 PM

PharoWeekly

[Ann] Large image generator

Hi

The consortium is starting a task force to make sure that people can work with large images. So we will be revisiting all the tools and internal logic during the coming months.

We need to generate rather large code base. We started with a simple large image generator. If you have ideas and want to contribute let us know.

https://github.com/tesonep/pharo-image-generator/

Consortium team

 

 

by Stéphane Ducasse at January 29, 2020 11:14 AM

January 27, 2020

PharoWeekly

[ann] ThreadedFFI v1.2.0

Pablo Tesone from the consortium announced threadedFFI v1.2.0

Version v1.2.0

Repository: pharo-project/threadedFFI-Plugin · Tag: v1.2.0 · Commit: d868e9b · Released by: tesonep

by Stéphane Ducasse at January 27, 2020 05:24 PM

Fun with images

https://80738163270632.blogspot.com/2018/10/pharo-script-of-day-proto-proto-image.html

Thanks Hernan

by Stéphane Ducasse at January 27, 2020 10:30 AM

January 26, 2020

Program in Objects

Extending the Smalltalk Syntax 3

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 3: Tagged Nodes

What do you do when you have to include JSON in a Smalltalk method? Something like this?

jsonCoordinates
  ^'{
      "latitude": 48.858093,
      "longitude": 2.294694
    }'

In other words, do you represent JSON data with plain strings? Wouldn’t it be nice to improve this? What if the compiler knew that this String should conform to a specific syntax? What if the Smalltalk browser knew how to format JSON strings? Or even color them for the sake of readability?

Before jumping to JSON strings, let’s step back and think of other cases that might be similar. Have you ever represented HTML as a plain String? What about CSS or even JavaScript? I’m sure you have faced situations where you inlined foreign code as a plain String in a method, keeping that information in your head rather than in the method, where it should naturally belong? Want to change this? Ok. Let’s do it.

Where to start? Here is the roadmap:

  1. Consider the introduction of tags for inlining foreign scripts.
  2. Introduce a new subclass of LiteralNode named TaggedNode .
  3. Consider the introduction of foreign parsers such as a JsonParser.
  4. Introduce a new class of AST node named ForeignNode.
  5. Process the body of the foreign script, according to its semantics.

Task 1: Smalltalk tags?


Before making a decision for tags, let’s see which other options do we have. In order to inline foreign scripts, we must tell the Smalltalk parser how to delimit them. There are several delimiters already taken in Smalltalk:

Can we think of any other? Backticks are tempting. The problem is that they would only work for a single semantics. Say we decide to delimit JSON using backticks; how would we delimit HTML or CSS or JavaScript or Assembly or C, should the future bring a need for any of them?

We want flexibility and that’s why tags are a good choice.
Using tags we will be able to inline foreign code like this:

jsonCoordinates
  ^<json>
    {
      "latitude": 48.858093,
      "longitude": 2.294694
    }
  </json>

And how do we make sure that tags do not confuse the Smalltalk parser? To answer this question think of all the places where $< is misplaced in regards to the Smalltalk syntax:

In other words, none of the following sequence of characters conforms to the Smalltalk syntax:

See? Every potential syntax error is an opportunity for extending the syntax!

Of course, angle brackets <...> are already legal in the Smalltalk syntax as pragma delimiters. But pragmas are illegal when placed in assignments, arguments and returns. To be valid, they must start a Smalltalk statement. And this is precisely why we will forbid tags at the beginning of statements and restrict them to assignments, arguments and returns.

Task 2: Add the class for tagged nodes


A tagged node is a way of delimiting foreign code and as such it is a new type of literal. So, add a subclass of LiteralNode named TaggedNode. This subclass will add the tag ivar that will link its instances to their specific meaning.

As we depticted above, instances of TaggedNode need to be instantiated by the parser in the following four cases:

This means that we need to modify essentially four methods so that they now check whether the next character is $<. If it is not, the original code regains control. Otherwise, the code branches to a new method that will scan the TaggedNode or fail (if there is no closing tag, etc.).

I’ve used the verb to scan because in order to form a TaggedNode, we will need to scan the input at the character level (usually the parser deals with tokens provided by the scanner).

When scanning the opening tag we will need to read the input until $> is reached (issuing and error if it isn’t). This will give us the value for the tag ivar of the TaggedNode. At this time we will also know that the closing tag should be '/', tag. So we can read the body of the foreign code until '</', tag, '>' is found (error if not).

At this point we are ready to

Task 3: Decide how to process the foreign script


Now we have access to the body of the TaggedNode. What do we do with it? Well, this depends on the semantics we want to give it. In the case of JSON, for instance, it would be enough to parse it using a JSON parser, and then format it using a JSON writer. We could also paint it with colors and emphases, so to make it look great in our environment.

In other cases, such as the one where the foreign code is Assembly, we could decide to go a step further and compile it into machine code. This will bring two capabilities: (1) Parsing and formatting/painting the Assembly source code and (2) Making the node answer with the corresponding machine code when the method is executed.

There are many other possibilities. In the case of JavaScript or any other programming language, we could decide to execute it on top of Smalltalk (at least up to some extent, this should be feasible).

On the other end of our wide horizon of possibilities there is one that consists in doing nothing, i.e, simply keeping the body as a String with no special semantics. This is useful for experimentation. For instance, if you plan to write a parser for inlining VBA code, before embarking in such a project, you might want to see how
the tagged code would look. You will need to format it yourself and will have no coloring available. However, it will bring a secondary benefit: you will not have to worry about duplicating embedded quotes (in VBA the quote is the comment separator).

The last case is the simpler one. Still, it requires the introduction of a new kind of literal node, which we will call StringNode. So, instead of keeping the body in the TaggedNode as a String, we will create a StringNode with the body as its value and will keep this new node as the value of the TaggedNode. This indirection will provide the flexibility we need for making free use of TaggedNode.

Note also that while making these decisions you should keep in mind that sometimes it is not necessary to implement a parser of the entire specification of the foreign language. For instance, if you will only deal with CTypes and C-Structures, you don’t need to parse arbitrary C, you just need to parse these declarations. The same might be true with other languages. Usually you will only inline a limited subset of them. The key here is to create the machinery that will make further enhancements easier and consistent.

Task 4: The foreign node


So far we have discussed two new nodes: TaggedNode and StringNode, both subclasses of LiteralNode. The ivar tag in TaggedNode holds the node’s tag string. Where we have to be careful is in deciding the contents of the value ivar because this is where the semantics enters the game.

Since we are planning for support of different languages, we will need a global Registry of available parsers/compilers. For instance, the package that loads the JSON parser will be able to register the <json> tag with the corresponding parser. Similarly for <html>, <css>, <asm>, <js>, <vba>, etc.

In this way, when the TaggedNode receives the #body: message with the foreign code as the argument, it will be able to enter the Registry with its tag and get the corresponding parser from there. If there is none, the TaggedNode will resort to StringNode, passing it the body and keeping this node in its value ivar.

TaggedNode >> body: aString
  value := Registry
    at: tag
      ifPresent: [:p | ForeignNode new parser: p]
      ifAbsent: [StringNode new].
  value value: aString

The ForeignNode will have two ivars: parser and ast. The latter is computed as follows:

ForeignNode >> value: aString
  ast := parser parse: aString

Task 5: Compile/Process


Now that we have all the pieces in place we can use them to, at least, format and/or color the foreign code. This is simply achieved by asking the ForeignNode, its ast or its formatted/colored representation. You could also provide more advanced featrues such as the ones we have mentioned in Task 3, or even take advantage of yet another technique that we will discuss in the next story which is Hybrid Compilation.

by smalltalkrenaissance at January 26, 2020 08:51 PM

January 25, 2020

Program in Objects

A Canticle for Smalltalk

A Smalltalk devotee created this lovely video:

“A new programming anthem for the future of software development.”

Indeed.

by smalltalkrenaissance at January 25, 2020 06:15 PM

PharoWeekly

ETS joining the consortium

The Pharo Consortium is very happy to announce that ÉTS Montréal has joined the Consortium as an Academic Member.

About
– ÉTS Montréal: https://www.etsmtl.ca/en/Home
– Pharo Consortium: http://consortium.pharo.org

The goal of the Pharo Consortium is to allow companies and institutions to support the ongoing development and future of Pharo.

Individuals can support Pharo via the Pharo Association: http://association.pharo.org

by Stéphane Ducasse at January 25, 2020 07:55 AM

January 23, 2020

Pierce Ng

Embedding Pharo in Pascal HOWTO - Linux

Pharo headless VM

First, build the Pharo headless VM, following the instructions in its README. When done, the VM files are in build/vm.

Pharo application image

Next, prepare the Pharo application image. Basically, start from a fresh Pharo 8.0 image and load the EmbeddedSupport code from Pablo's repository. The following snippet works, assuming you have cloned Pablo's repository locally; adjust the gitlocal path in the snippet, of course.

Metacello new
    baseline: 'EmbeddedSupport';
    repository: 'gitlocal:///home/pierce/src/st/pharo-vm-embedded-example/smalltalk-src';
    load.

NoChangesLog install.
NoPharoFilesOpener install.
PharoCommandLineHandler forcePreferencesOmission: true.

SmalltalkImage current snapshot: true andQuit: true

Windows resource

Then, embed the Pharo application image into a Windows resource file.

Here's a simplified version of resources.rc. This assumes you have named the image to be embedded Pharo.image.

300 RCDATA "Pharo.image"

Building a Windows resource file on Linux requires windres. On Ubuntu, this program is part of the package binutils-mingw-w64-x86-64.

Place the application Pharo.image and resources.rc in the same directory. Then,

% /usr/bin/x86_64-w64-mingw32-windres -i resources.rc -o resources.res
% ls -l
total 113488
-rw-r--r-- 2 pierce pierce 58098672 Jan 23 14:10 Pharo.image
-rw-r--r-- 2 pierce pierce       25 Jan 23 14:01 resources.rc
-rw-r--r-- 2 pierce pierce 58098736 Jan 23 14:10 resources.res

The output resources.res is the compiled resources file.

Pascal host program

Finally we are ready to build the Pascal host program. Move resources.res to this repository's embedded-cli directory. Also create a link to libPharoVMCore.so or make a copy of it. The directory's content is now as follows:

% ls -l
total 58736
-rwxr-xr-x 2 pierce pierce  3344760 Jan 23 09:26 libPharoVMCore.so*
-rw-r--r-- 2 pierce pierce 58098736 Jan 23 14:10 resources.res
-rw-r--r-- 4 pierce pierce     1945 Jan 23 12:55 runPharoEmbedded.pas
-rw-r--r-- 4 pierce pierce     3824 Jan 23 12:54 ulibEmbeddedImage.pas
-rw-r--r-- 5 pierce pierce     1045 Jan 22 15:42 ulibPharoVM.pas

At the start of runPharoEmbedded.pas, the line {$R resources.res} is the Pascal compiler directive to incorporate resources.res into the executable that is being built:

program runPharoEmbedded;

{$mode objfpc}{$H+}
{$R resources.res} { <= compiler directive to incorporate resources.res }
{$linklib m}

Build the host program - the Pascal compiler is its own make system and knows to compile the necessary source files. (The Pascal compiler also knows how to invoke windres to compile resources.rc into resources.res when so configured. I've done that part by hand since this is a HOWTO.)

% fpc runPharoEmbedded.pas
Free Pascal Compiler version 3.0.4 [2018/10/29] for x86_64
Copyright (c) 1993-2017 by Florian Klaempfl and others
Target OS: Linux for x86-64
Compiling runPharoEmbedded.pas
Compiling ulibPharoVM.pas
Compiling ulibEmbeddedImage.pas
Compiling resource runPharoEmbedded.or
Linking runPharoEmbedded
/usr/bin/ld: warning: link.res contains output sections; did you forget -T?
232 lines compiled, 0.4 sec

% ls -l runPharoEmbedded
-rwxr-xr-x 1 pierce pierce 58884328 Jan 23 14:22 runPharoEmbedded*

Much of the size of the executable is due to the embedded Pharo.image.

We'll run runPharoEmbedded in the headless VM build directory because Pharo's baked-in library lookup currently requires this. (More on this in a later post.) So move the program over.

% mv runPharoEmbedded ~/src/st/opensmalltalk-vm-pharo/build/vm
% cd ~/src/st/opensmalltalk-vm-pharo/build/vm
% ls -T 30
libB2DPlugin.so*             libgit2.so.0.25.1             libSDL2-2.0.so.0*      libssl.so*
libBitBltPlugin.so*          libgit2.so.25                 libSDL2-2.0.so.0.7.0*  libssl.so.1.0.0*
libcrypto.so.1.0.0*          libIA32ABI.so*                libSDL2.so*            libSurfacePlugin.so*
libDSAPrims.so*              libJPEGReaderPlugin.so*       libSecurityPlugin.so*  libTestLibrary.so*
libffi.so*                   libJPEGReadWriter2Plugin.so*  libSocketPlugin.so*    libUnixOSProcessPlugin.so*
libffi.so.7*                 libLargeIntegers.so*          libSqueakFFIPrims.so*  libUUIDPlugin.so*
libffi.so.7.1.0*             libLocalePlugin.so*           libSqueakSSL.so*       pharo*
libFileAttributesPlugin.so*  libMiscPrimitivePlugin.so*    libssh2.so*            runPharoEmbedded*
libFilePlugin.so*            libPharoVMCore.so*            libssh2.so.1*
libgit2.so                   libPThreadedPlugin.so*        libssh2.so.1.0.1*

Set up LD_LIBRARY_PATH. The first path segment is for the Pharo VM. The second is for libcairo2.so needed by the embedded Pharo.image - on Ubuntu, it lives in /usr/lib/x86_64-linux-gnu, which isn't in Pharo 8's current hardcoded lookup path. Then run the executable:

% export LD_LIBRARY_PATH=`pwd`:/usr/lib/x86_64-linux-gnu
% uname -a
Linux Otrus 4.15.0-74-generic #84-Ubuntu SMP Thu Dec 19 08:06:28 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
% ./runPharoEmbedded
lala

runPharoEmbedded

Ta da! "lala" is printed by Pharo.

by Pierce Ng at January 23, 2020 02:40 PM

Program in Objects

Extending the Smalltalk Syntax 2

by Leandro Caniglia
President of FAST (Fundación Argentina de Smalltalk)

Story 2: Supporting Pragmas

All Smalltalk dialects support pragmas. Here is one:

identityHash
  <primitive: Hash>
  ^0

However, not all of them support other types of pragmas, as Pharo does. So, let’s see what it would take to add support for them to our dialect.

Where to start? Here is the roadmap:

  1. Find the Smalltalk Parser in your system
  2. In the Smalltalk Parser find the place where primitives are parsed
  3. Modify the code that parses primitive declarations to recognize pragmas as well
  4. Find a place in the CompiledMethod to store the pragma annotation

Now let’s see some hints on how to accomplish the tasks above:

Task 1: Find the Smalltalk Parser


Look for selectors like parse, parse:, parseMethod:, parseLiteral:, etc. In some dialects the entry point to parsing services is the Compiler, so you can debug the initial steps required to compile any expression until you reach the parser. You can also check to see if there are implementors of parserClass.

Task 2: Find the parsing of primitive declarations


This is best accomplished by debugging the compilation of any method with a primitive declaration. Just create a primitive method and debug its compilation. After some few clicks you should get to the parser method that distinguishes it as primitive.

Task 3: Parse the pragma


For this task I would recommend starting simple. There will be time for further sophistication. In this case “simple” means: assume the pragma is just one token. Much like:

methodWithPragma
  <ourPragma>
  self doSomething; etc.

The existing code in the parser should complain if the token that comes after $< is not 'primitive:' (or friends). And this is precisely where we need to branch, save the token as our pragma and (in this first attempt) save it in the MethodNode. Note that there is no need to add a new ivar to MethodNode, we can save the token in the same ivar where primitives are held. After all, primitives are numbers while pragmas are not, so we will be able to distinguish between them.

Don’t forget to consume the closing $>, which should be the next token. Otherwise issue a parsing error.

Task 4: Save the pragma in the method


This step is actually not required as we could always resort to the AST for getting the pragma. One could argue that it would be too slow to parse the entire method every time we want to access its pragma, should it have any. However, since pragmas occur before any other statements, there is no need to build the entire AST. In fact, we would only need a service in the parser that will stop parsing right before the parsing of the method sentences starts.

However, if you insist on avoiding any parsing, when a new CompiledMethod is compiled, the MethodNode should somehow inject the pragma in it. One way to do this is to add the pragma as the first literal. A practice also known as stealing literals. However, if we steal the first literal, how are we going to tell whether this first literal is a pragma or not?

There might be several tricks for solving this. For instance, in most dialects 0 (the SmallInteger) is never saved in the literal frame. The reason is that there are special bytecodes to operate with it, so the constant 0 doesn’t need to go into the literal frame.

Therefore, we could add the pragma as the first literal, and then add 0 as the second. Thus, in order to check and retrieve the pragma of a method we would do the following:

CompiledMethod >> pragma
  literals := self literals.
  ^(literals size >= 2 and: [(literals at: 2) = 0])
    ifTrue: [literals at: 1]

Note that the method will answer with nil if it has no pragma.

Final words


To make sure that 0 doesn’t go to the literal frame in your dialect, evaluate the following:

CompiledMethod allInstances select: [:cm | cm literals includes: 0]

It should answer with an empty array. If not, you will need to find another trick (hint: add two 0s instead of one.)

The simple case I’ve depicted here is for adding support to unary pragmas. You might want to allow for binary and keyword pragmas as well. The idea is the same. Just keep reading more tokens in Task 3, until $> is reached. Take a look at how Pharo does this for inspiration, starting at RBParser >> parsePragma. Then adapt the idea to your case.

If you decide to add support for multiple pragmas, note that you will need to steal n + 1 literals rather than 2 as we did in Task 4. You only need to move the 0 right after the last injection in the literal frame.

by smalltalkrenaissance at January 23, 2020 11:45 AM

PharoWeekly

[ANN] SmalltalkCI now supports Pharo 9

Hi,
Just to let you know that smalltalkCI was updated to include Pharo 9, 32 and 64 bits as available versions to be used, and stable/alpha were updated to be Pharo 8 and Pharo 9.
Thanks Pablo Tesone & Fabio Niephaus for helping in getting this updates ready.
Regards,
Gabriel

by Stéphane Ducasse at January 23, 2020 07:54 AM

January 22, 2020

Pierce Ng

Cross Platform Embedded Pharo

Recently Pablo Tesone published an example of embedding a Pharo image as a Windows resource inside a program written in C.

Well, the excellent and, like Smalltalk, vastly underrated Free Pascal programming system has cross-platform support for handling Windows resources. With it, I have reimplemented Pablo's example: One source code building executables that run on Ubuntu 18.04 and MacOS Mojave. Not tested on Windows yet, but I'm sure it'll work there too. The three Pascal source files, including blank lines and comments, total 209 lines:

Repository on GH. More details to come.

by Pierce Ng at January 22, 2020 07:32 PM

PharoWeekly

[Ann] Pharo embedded in Pascal :)

Hi all,

Inspired by Pablo’s example, I’ve created a cross-platform
implemenation. Pablo’s example uses Pharo.image embedded as a Windows
resource. Well, the  excellent and, like Smalltalk, vastly underrated
Free Pascal has built-in cross platform support for Windows resources.
Basically I have implemented, in Pascal, FFI to libPharoVMCore and the
embedded image access routines for callback by libPharoVMCore, together
with a driver program to run the embedded Pharo.image.

Repository is here:

https://github.com/PierceNg/pharo-vm-embedded-pascal

More details to come in the repository.

Pierce

by Stéphane Ducasse at January 22, 2020 12:32 PM

January 21, 2020

PharoWeekly

[ann] Neat for Pharo

Today is the official release of the NEAT algorithm for Pharo
🙂

by Stéphane Ducasse at January 21, 2020 08:51 PM