Planet Squeak

blogs about Squeak, Pharo, Croquet and family
planet squeak - planet squeak es - planet squeak jp - planet croquet - planet squeak code - planet smalltalk

December 09, 2018

Pierce Ng

This blog now on HTTPS

This blog is now on HTTPS.

Setup:

Caddy is an open source HTTP/2 web server. caddy-docker-proxy is a plugin for Caddy enabling Docker integration - when an appropriately configured Docker container or service is brought up, caddy-docker-proxy generates a Caddy site specification entry for it and reloads Caddy. With Caddy's built-in Let's Encrypt functionality, this allows the new container/service to run over HTTPS seamlessly.

Below is my docker-compose.yml for Caddy. I built Caddy with the caddy-docker-proxy plugin from source and named the resulting Docker image samadhiweb/caddy. The Docker network caddynet is the private network for Caddy and the services it is proxying. The Docker volume caddy-data is for persistence of data such as cryptographic keys and certificates.

version: '3.6'

services:

  caddy:
    image: samadhiweb/caddy
    command: -agree -docker-caddyfile-path=/pkg/caddy/caddyfile -log=/var/log/caddy/caddy.log
    ports: 
      - "80:80"
      - "443:443"
    networks: 
      - caddynet
    volumes:
      - type: bind
        source: /var/run/docker.sock
        target: /var/run/docker.sock
      - type: bind
        source: /pkg/caddy
        target: /pkg/caddy
      - type: volume
        source: caddy-data
        target: /root/.caddy
      - type: bind
        source: /var/log/caddy
        target: /var/log/caddy
    restart: unless-stopped

networks:
  caddynet:
    name: caddynet
    external: true

volumes:
  caddy-data:
    name: caddy-data
    external: true

Here's the docker-compose.yml snippet for the blog engine:

services:
  scms1app:
    image: samadhiweb/scms1app
    ports: 
      - "8081:8081"
    networks:
      - caddynet
    volumes:
      - type: bind
        source: /pkg/smallcms1/config.json
        target: /etc/smallcms1/config.json
      - type: volume
        source: smdw-content
        target: /pkg/cms
    labels:
      - "caddy.address=www.samadhiweb.com"
      - "caddy.targetport=8081"
      - "caddy.targetprotocol=http"
      - "caddy.proxy.header_upstream_1=Host www.samadhiweb.com"
      - "caddy.proxy.header_upstream_2=X-Real-IP {remote}"
      - "caddy.proxy.header_upstream_3=X-Forwarded-For {remote}"
      - "caddy.tls=email-address@samadhiweb.com"
      - "caddy.log=/var/log/caddy/www.samadhiweb.com.access.log"
    ulimits:
      rtprio:
        soft: 2
        hard: 2
    restart: unless-stopped

networks:
  caddynet:
    name: caddynet
    external: true

volumes:
  smdw-content:
    name: smdw-content
    external: true

Of interest are the caddy.* labels from which caddy-docker-proxy generates the following in-memory Caddy site entry:

www.samadhiweb.com {
  log /var/log/caddy/www.samadhiweb.com.access.log
  proxy / http://<private-docker-ip>:8081 {
    header_upstream Host www.samadhiweb.com
    header_upstream X-Real-IP {remote}
    header_upstream X-Forwarded-For {remote}
  }
  tls email-address@samadhiweb.com
}

Also note the ulimits section, which sets the suggested limits for the Pharo VM heartbeat thread. These limits must be set in the docker-compose file or on the docker command line - copying a prepared file into /etc/security/limits.d/pharo.conf does not work when run in a Docker container.

    ulimits:
      rtprio:
        soft: 2
        hard: 2

by Pierce Ng at December 09, 2018 07:05 PM

December 05, 2018

PharoWeekly

Dispatching tasks to multiple VMs

https://blog.lxsang.me/post/id/25/Dispatching-tasks-to-multiple-Pharo-VMs

by Stéphane Ducasse at December 05, 2018 05:23 PM

Ecole des Mines

Dispatching tasks to multiple Pharo VMs using SystemProcess

Doing a research work in robotic domain using Pharo as a prototype and implementation tool (via PhaROS) is a whole new experience. It is quite impressive to see how quick an implementation idea becomes a working prototype/solution in Pharo thanks to its productive development environment. Most of my robotic applications are critical tasks which require… Continue reading

by noury at December 05, 2018 03:54 PM

December 04, 2018

PharoWeekly

[ann] PetitParser on github

Hello,

I just finished the migration of PetitParser (1) on Github: https://github.com/moosetechnology/PetitParser <https://github.com/moosetechnology/PetitParser> .

I created a release v1.9.2 which mirrors latest release of Smalltalkhub.

Cheers,

Julien

by Stéphane Ducasse at December 04, 2018 02:44 PM

[ann] Ethereum driver

Hi All!

I have being working on releasing versions (i should do it more often).
 I am happy to announce Fog v1.0.0-beta. I think I found a API that i like
enough. So, i took the courage to pass to 1.

 Thew new version changes many things:  Redefines the roll of the session,
it minimize the roll of the connection, it cleans up the code, it defines a
more verbose but less magic API. It generates proxies for the contracts.

It supports now all the types including: maps and arrays.

Goodnight :)


Santiago


Fog

Pharo Ethereum Driver

Fog is a library that allows the user to connect to an ethereum based
blockchain data base.

Is based on the Javascript canonical implementation done by Ethereum
community Ethereum Javascript API
<https://github.com/ethereum/wiki/wiki/JavaScript-API>.
<https://github.com/smartanvil/Fog#dependencies>
Download code <https://github.com/smartanvil/Fog#iceberg--baseline>Iceberg
/ Baseline

Metacello
	new
	baseline: 'Fog';
	repository: 'github://sbragagnolo/Fog/src';
	load.

by Stéphane Ducasse at December 04, 2018 12:38 PM

December 02, 2018

Clement Bera

VM Learning: Call Stack Management (V2, C version)

Hi all,

In 2016, I published a first article [1], which lead to a booklet [2] to help people learning about Call Stack Management. In short, the book includes descriptions and exercises to implement interpreters with an increasing level of difficulty. It allows the reader to understand concepts such as calling conventions, access to variables relatively to the frame pointer and stack pointer, stack pages and closure implementation.

This year, I have the chance to be a post-doc at the VUB with Elisa Gonzalez Boix. Among lots of interesting work we do together, she is teaching to Bachelor 2 x86 calling conventions and call stack management. It seems the existing practical work covered different aspects of the lecture, but not really calling conventions. The practical works associated with this lecture have to be in C, hence I ported one of the interpreter of the booklet (the one with an infinite stack but no closures) to C. With Maarten Vandercammen, the three of us built an assignment out of the C interpreter and taught it in a 2 hours practical work to Bachelor 2 students. Most students were able to implement all the given instructions of the interpreter (called stack machine in the assignment) in 2 hours. A very small subset of them made it to the “Going Further” section. Overall, the assignment went really well.

You can download the contents of the assignment here [3].

I was not allowed to share publicly a solution for the students not to cheat (they will do it next year again), but if you know me personally or request it with proper motivation I may share it. There is nothing fancy about it though, you can just implement the solution yourself, Bachelor 2 students do it in only 2 hours. I think in total only 21 lines of C code has to be written or something like that.

Difference with the original booklet

The C version is easier to implement that the Smalltalk version. It is not easier because C is easier to deal with, it is easier because the assignment covers only a subset of the booklet, but covers the most essential content.

In Smalltalk the language implemented (SmiLang) is a Smalltalk embedded DSL, in C it is a C embedded DSL. To stick closer to the C language, I did not implement metadata around functions as in Smalltalk. Function * in C is a pointer to the first instruction of the function, not to a struct which includes the number of arguments of the function and so on. This means that some instructions (PushArg, ReturnTop) had to be modified to encode extra/different information. In addition, instruction pointer, stack pointer and frame pointer are direct pointers inside data structures, and not indexes relative to the beginning of data structures as in Smalltalk. Lastly, in Smalltalk, IDE extensions are provided through GTInspector to support the work of the students, while in C debugging functions which print in the command line are provided instead.

Let me know if you questions or suggestions!

[1] https://clementbera.wordpress.com/2016/04/10/vm-learning-call-stack-management/
[2] https://files.pharo.org/books-pdfs/booklet-CallStack/2018-01-23-CallStack.pdf
[3] https://www.dropbox.com/s/a9cw1xkfyxrtcq6/SmiLang.zip?dl=0

by Clement Bera at December 02, 2018 05:35 PM

November 30, 2018

PharoWeekly

[Ann] Teapot on github

Hi,

I just want to let you know that the source code of Teapot was moved from
Smalltalkhub to GitHub.

The new repository is located at:

https://github.com/zeroflag/teapot <https://github.com/zeroflag/teapot>  

Attila

by Stéphane Ducasse at November 30, 2018 07:34 PM

November 29, 2018

PharoWeekly

Writing baselines… in Pharo 70

Great resources

https://github.com/pharo-tips-and-tricks/pharo-tips-and-tricks/blob/master/General/Baselines.md

Thanks for writing it.

by Stéphane Ducasse at November 29, 2018 10:27 PM

November 26, 2018

PharoWeekly

[ANN] JSON-RPC for Pharo

https://github.com/juliendelplanque/JRPC

JSON-RPC 2.0 implementation for Pharo Smalltalk

by Stéphane Ducasse at November 26, 2018 11:16 AM

November 24, 2018

PharoWeekly

[Ann] JSONSchema for Pharo

JSONSchema
===========

This is an implementation of JSON Schema for the pharo language. It is used to define the structure and values of a JSON string and to validate it. The schema itself can be externalized for being consumed by a third party.

I like to announce the availability of a JSON schema implementation for pharo. As part of my implementation of OpenAPI (which is to be released a bit later) I factored out the JSON schema part into its own repository because I think it is useful. I release it even it is not really finished. Code is mostly undocumented and a lot of features are missing from the full spec. I will improve it slowly and add features as I need them or they being requested

Hope you like it!

Norbert 

====

The documentation so far (from https://github.com/zweidenker/JSONSchema <https://github.com/zweidenker/JSONSchema>)

It can be loaded by downloading it in pharo via

  Metacello new
    repository: 'github://zweidenker/JSONSchema';
    baseline: #JSONSchema;
    load

Defining a schema
-------------------------

These are the expression to create a schema model inside pharo.

schema := {
  #name -> JSONSchema string.
  #dateAndTime -> (JSONSchema stringWithFormat: 'date-time').
  #numberOfPets -> JSONSchema number } asJSONSchema.


defines as schema that can parse the following JSON:

jsonString := '{
  "name" : "John Doe",
  "dateAndTime" : "1970-01-01T14:00:00",
  "numberOfPets" : 3
}'.

Reading/Writing a value using a schema
------------------------------------------------------

To parse the value from JSON we only need to invoke:

value := schema read: jsonString

The object in value will have name as a string, dateAndTime as a DateAndTime object and numberOfPets as a SmallInteger object.

The schema can also be used to write out the value as JSON. This is especially useful if we want to ensure that only valid JSON is written. For this invoke

jsonString := schema write: value.

Serialize/Materialize a schema
----------------------------------------

Addtionally to reading and writing objects a schema can be serialized to string.

schemaString := NeoJSONWriter toStringPretty: schema.

gives

{
	"type" : "object",
	"properties" : {
		"name" : {
			"type" : "string"
		},
		"numberOfPets" : {
			"type" : "number"
		},
		"dateAndTime" : {
			"type" : "string",
			"format" : "date-time"
		}
	}
}


If we would get a schema as string we can instantiate by invoking

schema := JSONSchema fromString: schemaString.

Nested schemas
-----------------------

Schemas can be nested in any depth. And it can be specified by using the literal Array syntax.

schema := {
  #name -> JSONSchema string.
  #address -> {
    #street -> JSONSchema string.
    #number -> JSONSchema number
  } } asJSONSchema

Constraints
---------------

JSON Schema has a defined set of constraints that can be specified. E.g. for a number the inerval of the value can be specified by

numberSchema := JSONSchema number.
numberSchema interval
  minimum: 1;
  exclusiveMaximum: 100

constraining the number value to be greater or equal to 1 and smaller than 100.

by Stéphane Ducasse at November 24, 2018 08:07 AM

Hernan Morales

Pharo Script of the Day: Super easy meme generation

Couldn't resist sharing this one:

ZnClient new
logToTranscript;
url: 'https://api.imgflip.com/';
addPath: 'caption_image';
formAt: 'template_id' put: '95158268';
formAt: 'username' put: '...';
formAt: 'password' put: '...';
formAt: 'text0' put: 'Just Zinc';
formAt: 'text1' put: 'HTTP Components';
accept: 'application/json';
contentReader: [ : entity |
ZnEasy getJpeg: (((NeoJSONReader fromString: entity contents) at: #data) at: #url) display ];
post.

To save you from generating a user in imgflip API, this is the resulting output:


by Hernán (noreply@blogger.com) at November 24, 2018 01:00 AM

November 23, 2018

PharoWeekly

Zinc HTTP Components version 3.0.1

I finally merged and synced all Zinc HTTP Components repos and did a full release on Pharo versions 3 to 7.

  https://github.com/svenvc/zinc/releases/tag/v3.0.1

Main README was updated

  https://github.com/svenvc/zinc

Green build for Pharo 7 RC1 (linux 64-bit) using BaselineOfZincHTTPComponents

  https://travis-ci.org/svenvc/zinc

Green builds for Pharo 3, 4, 5 and 6 using ConfigurationOfZincHTTPComponents (#stable 3.0.1)

  https://ci.inria.fr/pharo-contribution/job/ZincHTTPComponents/

If there are any problems or issues, don't hesitate to ask.

Sven

by Stéphane Ducasse at November 23, 2018 09:53 PM

November 22, 2018

PharoWeekly

[ann] New Pharo success story

http://pharo.org/success/RMapViewer

Well done Tomohiro Oda and others !!!

 

 

by Stéphane Ducasse at November 22, 2018 08:10 PM

November 21, 2018

PharoWeekly

Pharo IoT hackathon!

by Stéphane Ducasse at November 21, 2018 04:55 PM

November 17, 2018

Hernan Morales

Pharo Script of the Day: DesignInfo

Hi there.

Not so much of a script today. Continuing a little bit on the SLOC subject, yesterday I kind of resurrected on GitHub a package called DesignInfo. This package is based purely on Morphic (so it is possible even to run it without Roassal or Moose) and takes every package in a Pharo image, and count the lines of code (LOC, as provided by RPackage), along with other metrics like:

  • Lines of production code
  • Lines of test code
  • Percentage of production code
  • Percentage of test code

You can have a look with the following expressions:

DesignInfo linesOfCodeSortedMorph openInWorld.
DesignInfo linesOfCodeMorph openInWorld.

A histogram is displayed where red bars represents production code and blue bars represents test code. See two screenshots of what the package provides:

by Hernán (noreply@blogger.com) at November 17, 2018 09:01 PM

November 14, 2018

Torsten Bergmann

What FP can learn from Smalltalk

by Torsten (noreply@blogger.com) at November 14, 2018 04:32 PM

November 13, 2018

PharoWeekly

Better management of encoding of environment variables

Hi all,

Thanks Ben for reading.

For those wanting a follow up, I’ve proposed this pull request:
https://github.com/pharo-project/pharo/pull/1980.
I’m still working on avoiding dependencies against UFFI, fixing one other
test.
This is however almost finished, and given that I had to adapt the
original *abstract
proposal* to fit the real system, here is an updated version:

API Proposal for OSEnvironment and friends
=========================

OSEnvironment is the common denominator for all platforms. They should
implement at least the following messages with the following semantics:

– *at: aVariableName [ifAbsent:/ifAbsentPut:/ifPresent:ifAbsent:]*

Gets the String value of an environment variable called `aVariableName`.
It is the system reponsibility to manage the encoding of *both arguments
and return values*.

– *at: aVariableName put: aValue*

Sets the environment variable called `aVariableName` to value `aValue`.
It is the system reponsibility to manage the encoding of *both arguments
and return values*.

– *removeKey: aVariableName*

Removes the environment variable called `aVariableName`.
It is the system reponsibility to manage the encoding of *both arguments
and return values*.

API Extensions for *Nix Systems (OSX & Linux)
=========================

Since *Nixes environment variables are binary data that could be encoded in
any encoding, the following methods provide more flexibility to access such
data in the encoding of the choice of the user, or even in binary form.

– *at: aVariableName encoding: anEncoding
[ifAbsent:/ifAbsentPut:/ifPresent:ifAbsent:/put:] / removeKey:**
aVariableName
encoding: anEncoding*

Variants of the common API from OSEnvironment.
The encoding used as argument will be used to encode/decode *both arguments
and return values*.

– *rawAt: anEncodedVariableName encoding: anEncoding
[ifAbsent:/ifAbsentPut:/ifPresent:ifAbsent:/put:] / removeRawKey:*
*anEncodedVariableName*

Variants of the common API from OSEnvironment.
These methods assume arguments and return values are encoded/decoded by the
user, so no marshalling or decoded is done by it.

Rationale
=========================

– Encoding/Decoding should be applied not only to values but to
variables names too. In most cases Ascii overlaps with utf* and Latin*
encodings, but this cannot be simply assumed.
– Windows requires calling the right *Wide version of the functions from
C, plus the correct encoding routine. This could be implemented as an FFI
call or by modifying the VM to do it properly instead of calling the Ascii
version.
– Unix FileSystems and environment variables could mix strings in
different encodings, thus the flexibility added by the low level *Nix
extensions.

Other Implementation Details
=========================

– VM primitives returning paths Strings should be carefuly managed to
decode them, since they are actually C strings (so byte arrays) disguised
as ByteStrings.
– Similar changes had to be applied to correctly obtain the current
working directory in case it is a wide string.

 

Guille

by Stéphane Ducasse at November 13, 2018 06:55 PM

November 11, 2018

PharoWeekly

New random generator: WELL512 PRNG

Hi,

By accident I came across a pseudo random number generator that I never heard off before. It is supposed to be pretty good and had a very simple implementation. So I ported it to Pharo.

<class comment>

I am RandomWELL512, a random number generator.

I use the PRNG (Pseudo Randon Number Generator) WELL (Well equidistributed long-period linear) with a 512 bit state.

  https://en.wikipedia.org/wiki/Well_equidistributed_long-period_linear
  http://www.iro.umontreal.ca/~lecuyer/myftp/papers/wellrng.pdf

Implementation algorithm (See #nextUInt32)

  http://www.lomont.org/Math/Papers/2008/Lomont_PRNG_2008.pdf (Chris Lomont, www.lomont.org)

Usage

  RandomWELL512 new in: [ :r | (1 to: 10) collect: [ :i | r nextUInt32 ] ]. 

  RandomWELL512 new useUnixRandomGeneratorSeed; in: [ :r | (1 to: 10) collect: [ :i | r next ] ]. 

  RandomWELL512 new in: [ :r | (1 to: 10) collect: [ :i | r nextInt: 1000 ] ].

  RandomWELL512 new in: [ :random | | count all |
    random useUnixRandomGeneratorSeed.
    count := 1024 * 1024.
    all := Array new: count streamContents: [ :out |
      count timesRepeat: [ out nextPut: random next ] ].
    { all min. all max. all average. all stdev } ].

  [ RandomWELL512 new in: [ :random | 1 to: 1e6 do: [ :i | random next ] ] ] timeToRun.

Note that you should create one instance, seed it properly, and keep using it.

</class comment>

It is acceptably fast, generating 1M [0,1) Floats in about 0.1s. I compared the output with a fixed initial state to the C code that I started from and I got the same numbers. Maybe some people find this interesting.

I attached a file out.

Sven

by Stéphane Ducasse at November 11, 2018 08:53 AM

Zn nicer API

Hi,

I added a new convenience method to Zinc HTTP Components: ZnClient>>forJsonREST. This configures a ZnClient (HTTP client) to talk to standard JSON REST web services. Here are a couple of examples:

ZnClient new
  forJsonREST;
  get: 'https://jsonplaceholder.typicode.com/users'.

What #forJsonREST does is 3 things: set the 'Accept' header to 'application/json', install a #contentReader that parses incoming JSON as well as a #contentWriter that generates JSON.

ZnClient new
  systemPolicy;
  forJsonREST;
  url: 'https://jsonplaceholder.typicode.com/posts';
  contents: { #foo->1. #bar->2 } asDictionary;
  post.

As you can see, the full ZnClient API can be combined when needed.

ZnClient new
  forJsonREST;
  post: 'https://jsonplaceholder.typicode.com/posts' 
  contents: (NeoJSONObject new foo: 1; bar: 2; yourself).

#post:contents: combines separate #url: #contents: and #post message.

#forJsonREST uses NeoJSON[Object|Writer] if found, else STONJSON. If both are missing, this results in an error.
	
ZnClient new
  systemPolicy;
  forJsonREST;
  url: 'http://easy.t3-platform.net/rest/geo-ip';
  queryAt: #address put: '81.83.7.35';
  get.

Finally, here is a more sophisticated example, doing a DNS request over HTTPS:

ZnClient new
  systemPolicy;
  beOneShot;
  forJsonREST;
  accept: 'application/dns-json';
  url: 'https://cloudflare-dns.com/dns-query';
  queryAt: #name put: 'stfx.eu';
  queryAt: #type put: #AAAA;
  get.

Note that in most cases, you will configure one client to a specific endpoint and keep on reusing it. At one point in time it might be good to #close the client (although that happens on finalise as well). For single requests, you can use #beOneShot.

All this can be found in #bleedingEdge (HEAD). There are unit tests as well.

Sven

by Stéphane Ducasse at November 11, 2018 08:50 AM

November 09, 2018

Clement Bera

64 bits Immediate Floats

Hi all,

In this post I will try to discuss some inner details of OpenSmalltalk-VM immediate floats. Immediate floats are present only in 64 bits hence I won’t talk about 32 bits VM in the whole blog post. In addition, OpenSmalltalk-VM supports only double precision IEEE floating pointer, hence I won’t discuss single precision IEEE floating pointer.

Immediate objects

OpenSmalltalk-VM uses an immediate object scheme to represent object oriented pointers (oop) in memory. Basically, due to 64 bits alignment, the last 3 bits of all pointers to objects are 000. This is abused to encode in the oop itself specific objects, in our context, SmallIntegers, Characters and ImmediateFloats. This optimization allows to save memory and to improve performance by avoiding boxing allocation for common arithmetic operations. The last 3 bits of an oop are called a tag. The Immediate Float tag is 100 (4 in decimal). Objects encoded directly in the oop are in our terminology called immediate objects.

Immediate

Immediate floats

OpenSmalltalk-VM and its clients use the double precision IEEE format to represent floating pointers, supported by most modern hardware.

doubleIEEE.png

The key idea to the immediate float design is to use an immediate representation of double precision floats to avoid boxing and save memory, while still being 100% compatible with the IEEE double precision format (Customers requirement).

Therefore, in 64 bits, OpenSmalltalk-VM use two implementations for floats. The most common floats are represented with immediate floats, where 3 bits of the exponents are abused to encode the tags. The rest of the floats are represented as boxed floats.

ImmFloat

By design, immediate floats occupy just less than the middle 1/8th of the double range. They overlap the normal single-precision floats which also have 8 bit exponents, but exclude the single-precision denormals (exponent-127) and the single-precision NaNs (exponent +127). +/- zero is just a pair of values with both exponent and mantissa 0.

So the non-zero immediate doubles range from
+/- 0x3800,0000,0000,0001 / 5.8774717541114d-39
to +/- 0x47ff,ffff,ffff,ffff / 6.8056473384188d+38

Encoding and decoding

The encoded tagged form has the sign bit moved to the least significant bit, which allows for faster encode/decode because offsetting the exponent can’t overflow into the sign bit and because testing for +/- 0 is an unsigned compare for <= 0xf.

So given the tag is 4, the tagged non-zero bit patterns are
0x0000,0000,0000,001[c(8+4)]
to 0xffff,ffff,ffff,fff[c(8+4)]
and +/- 0d is 0x0000,0000,0000,000[c(8+4)]

Decoding of non-zero values in machine code is performed as follow:

Decoding.png

Encoding of non-zero values in machine code is performed as follow:

Encoding.png

Reading floats in general is fairly easy, the VM checks the class index, if the class index of immediate float is present, then the float is decoded from the oop, if the boxed float class index is present, the float is read from the boxed object.

Each primitive operation (arithmetic, comparison, etc.) has now to be implemented twice, once in both classes, where the first operand is expected to be an instance of the class where it is installed. In Smalltalk float primitive operations succeed if the second operand is one of the 2 float classes or a SmallInteger. It fails for large integers and arbitrary objects, in which case the VM takes a slow path to perform correctly the operation.

At the end of arithmetic operations, the resulting float has to be converted back from the unboxed format to either an immediate float or a boxed float. To do so, the VM checks the exponent of the float against the smallFloatExponentOffset, 896. 896 is 1023 – 127, where 1023 is the mid-point of the 11-bit double precision exponent range, and 127 is the mid-point of the 8-bit SmallDouble exponent range. If the exponent is in range, it can be converted to an immediate float. If not, one needs to check if the float is +/- 0, in which case it can still be converted to an immediate float, else it has to be converted to a boxed float. The code looks like that in Slang:

^exponent > self smallFloatExponentOffset
ifTrue: [exponent <= (255 + self smallFloatExponentOffset)]
ifFalse:
[(rawFloat bitAnd: (1 << self smallFloatMantissaBits - 1)) = 0
ifTrue: [exponent = 0]
ifFalse: [exponent = self smallFloatExponentOffset]]

x86_64 encoding/decoding

To conclude the post, here are the instructions generated in x86_64 to encode immediate floats. I put the instruction so that you can see how to encode efficiently using the theoretical design from the figures above and in addition quick checks for +/- 0.

000020da: rolq $1, %r9 : 49 D1 C1
000020dd: cmpq $0x1, %r9 : 49 83 F9 01
000020e1: jbe .+0xD (0x20f0=+@F0) : 76 0D
000020e3: movq $0x7000000000000000, %r8 : 4D B8 00 00 00 00 00 00 00 70
000020ed: subq %r8, %r9 : 4D 2B C8
000020f0: shlq $0x03, %r9 : 49 C1 E1 03
000020f4: addq $0x4, %r9 : 49 83 C1 04

Decoding is easier:

00002047: movq %rdx, %rax : 48 89 D0
0000204a: shrq $0x03, %rax : 48 C1 E8 03
0000204e: cmpq $0x1, %rax : 48 83 F8 01
00002052: jle .+0xD (0x2061=+@61) : 7E 0D
00002054: movq $0x7000000000000000, %r8 : 4D B8 00 00 00 00 00 00 00 70
0000205e: addq %r8, %rax : 49 03 C0
00002061: rorq $1, %rax : 48 D1 C8
00002064: movq %rax, %xmm0 : 66 48 0F 6E C0

Let me know if you have any question or you want me to expand this post with something else.

Note: Part of the blog post was extracted from the SpurMemoryManager class comment on immediate float, I thank Eliot Miranda and other OpenSmalltalk-VM contributors for writing it.

by Clement Bera at November 09, 2018 07:04 PM

November 07, 2018

PharoWeekly

[Ann] v1.4.0 of Iceberg

Hello!

This week we are releasing the version v1.4.0 of Iceberg.
(https://github.com/pharo-vcs/iceberg/releases/tag/v1.4.0)

This version is available in the latest Pharo 7.

This release fixes a bug introduced in v1.3. It also add new features
in the repository view, add some cleaning and also re-introduce a
feature that was lost.

Thanks to all contributors.

Enjoy!

Full changelog:

Bugfixes

#1068 'There is no associated repository configured.' warning on right
clicking missing repository

Features

#1077 Repository view: Allow to collapse branches/remotes/tags trees
#847 Move tags under remotes in Repository view
#1070 set upstream if missing

Cleanups

#1066 Pharo 7: PackageManifest subclasses should be packaged with "Manifest"
#1015 Replace usages of Glamour in the Github Plugin
#1063 1061-Introduce-iconNamed-in-IceDefinition-and-IceTipModel-and-remove-all-the-terrible-Smalltalk-ui-icons

by Stéphane Ducasse at November 07, 2018 09:10 PM

[ann] Pharo 7.0.0-rc1

Greetings!

I’m announcing today we reach Pharo 7.0.0-rc1!

This is the first step to release a definitive version, and while we will continue integrating bug fixes, API change Pull Requests will be delayed until the open of Pharo 8.0.0 development.
Now, you would wonder what is the ChangeLog of this release… and answer is we still do not have one (btw, we should find a way to automate this).

Anyway… we are very close to release now 🙂

Please download, test, report issues.

by Stéphane Ducasse at November 07, 2018 10:44 AM

November 06, 2018

PharoWeekly

[Ann] release v1.3 of Iceberg

Hello!

This week we are releasing the version v1.3 of Iceberg.
(https://github.com/pharo-vcs/iceberg/releases/tag/v1.3.0
+https://github.com/pharo-vcs/iceberg/releases/tag/v1.3.1)

This version will be available after we merge this PR:

https://github.com/pharo-project/pharo/pull/1951

This release contains some new features such as the support of self
hosted gitlab, integration with github, etc.
It also contains multiple bug fixes, cleanups and enhancements.

On the CI part, Guille made the Appveyor build green! This will
increase the stability of the windows support.

Thanks to all contributors.

Enjoy!

Full changelog:

Features

#1021 Self hosted gitlab support
#1027 Improved new tag dialog to generate semantic versionning tags
#1044 Show a button “View on Github” when creating a PR
#1008 Add “create branch from GitHub issue” option
#1048 Add commands to open remotes on Github
#1010 Add menu entry in extras to force calculate diff

Bug corrections

#975 Metacello asks too many times what to install when there are
conflicting versions
#980 Iceberg should Identify better the packages and the normal files
#982 The Edit Project should have a Warning if it will affect the packages
#986 Iceberg does not realize changes in extended classes
#999 Pulling and pushing to a gitolite server asks password
#984 Conversion to Tonel generates corrupted .properties
#1041 Filter in repository view don’t work with capital letters
#1019 Metacello Integration leaves Monticello leftover repositories
#859 Creating a branch and pushing does not sets the upstream
#1043 Packages starting with lowercase not recognized
#991 Error on right click in the project editon wizard
#775 Reviewing a PR is broken
#1036 Debugger if we try to merge without selecting a branch
#1064 Fix failing tests regarding clean code in Pharo

Enhancements

#988 Iceberg should load the packages in a single MCLoader (This will
make the loads of packages atomic)
#1001 Use “instance creation” instead of “instance-creation” for
method protocol name
#1004 Use displayScaleFactor in UI
#977 Add ToolTip help to the Commands
#1030 Better support for binary files
#1034 SSH passphrase is now hidden

Cleanups

#1018 Iceberg UI relies on deprecated classes from Spec and Commander
#1051 Clean useless huge hierarchy in Github plugin UI

Infrastructure Enhancements

#1023 Fix CI for windows

by Stéphane Ducasse at November 06, 2018 07:00 PM

November 03, 2018

Hernan Morales

Pharo Script of the Day: One-liner identity matrix

Check how easy is to create an identity matrix in Pharo, in just one line: require matrix size from user and print the output:

(Array2D identity: (UIManager default request: 'Enter size of the matrix:') asInteger) asString

by Hernán (noreply@blogger.com) at November 03, 2018 08:08 PM

November 02, 2018

PharoWeekly

[Pharo-dev] [ANN] The STON Specification

Hi,

Since there can never be enough documentation I finally took some time to write a more formal description of STON as a data format.

  https://github.com/svenvc/ston/blob/master/ston-spec.md

The idea is to let this stabilise a bit and to then update the two other documents describing STON, where necessary:

  https://github.com/svenvc/ston/blob/master/ston-paper.md
  https://ci.inria.fr/pharo-contribution/job/EnterprisePharoBook/lastSuccessfulBuild/artifact/book-result/STON/STON.html

Also, the latest changes in STON have to make their way to the Pharo image as well.

  https://github.com/svenvc/ston

All feedback is welcome.

Sven

by Stéphane Ducasse at November 02, 2018 09:58 AM

Torsten Bergmann

Dynamic creation of compiler plugins in Pharo

A new class (merged this week into Pharo 7) called "OCCompilerDynamicASTPlugin" allows for the dynamic creation of compiler plugins


Object compiler
addPlugin:
(OCCompilerDynamicASTPlugin
newFromTransformBlock: [ :ast | (RBParseTreeRewriter replaceLiteral: 42 with: 'meaning of life') executeTree: ast. ast. ]
andPriority: 0
);
evaluate: '42'.
This would give 'meaning of life'

by Torsten (noreply@blogger.com) at November 02, 2018 09:46 AM

October 30, 2018

Hernan Morales

Pharo Script of the Day: Mass image format conversion from PNG to JPEG

You might find useful the following code to convert a whole directory of image in PNG format to JPEG:

(FileSystem disk workingDirectory filesMatching: '*.png') do: [ : pngFile |
pngFile asFileReference binaryReadStreamDo: [ : stream |
PluginBasedJPEGReadWriter
putForm: (PNGReadWriter formFromStream: stream)
onFileNamed: pngFile withoutExtension , 'jpg' ] ]
displayingProgress: 'Converting images to JPG...'.

by Hernán (noreply@blogger.com) at October 30, 2018 04:20 AM

October 26, 2018

Hernan Morales

Pharo Script of the Day: Text analysis using tf-idf

Today's snippet takes a natural language text as input (a.k.a. the Corpus) where each line is considered a different document, and outputs a matrix of term documents with word mappings and frequencies for the given documents. This is also known as tf-idf, a distance metric widely used in information retrieval and provides the relevance or weight of terms in a document.

Why is not this just simple word counting?

If you increase relevance proportionally to word count, then all your query results will have words like "the" as the most relevant in the whole set of documents (or even in a single document), as it is a very common word. So you would need to decrease count for these common words, or increase count for "rare" words to get their relevance. This is where IDF (inverse document frequency) comes into play. With IDF you count documents, so you will assign low score to terms appeared in a lot of documents, then increasing the divider and decreasing relevance.

Finally, Stop words are removed and stemming is performed to reduce words with the same root.

First of all, you can install Moose-Algos (with some needed Hapax classes in a clean Pharo image by evaluating:


Metacello new
configuration: 'MooseAlgos';
smalltalkhubUser: 'Moose' project: 'MooseAlgos';
version: #development;
load.
Gofer it
smalltalkhubUser: 'GustavoSantos' project: 'Hapax';
package: 'Hapax';
package: 'Moose-Hapax-VectorSpace';
load.

Then you can execute the script:

| corpus tdm documents |
corpus := MalCorpus new.
documents := 'Julie loves me more than Linda loves me
Jane likes me more than Julie loves me'.
documents lines doWithIndex: [: doc : index |
corpus
addDocument: index asString
with: (MalTerms new
addString: doc
using: MalCamelcaseScanner;
yourself)].
corpus removeStopwords.
corpus stemAll.
tdm := HapTermDocumentMatrix on: corpus.
tdm.

by Hernán (noreply@blogger.com) at October 26, 2018 02:30 AM

October 25, 2018

Hernan Morales

Pharo Script of the Day: Count lines of code

Lines of code, LOC, SLOC, ELOC... one the simplest and metrics around, and we could find the method with the most LOC in the image with just one line of code (tested in Pharo 6.1):

SystemNavigation default allMethods 
collect: [ : m | m -> m linesOfCode ]
into: (SortedCollection sortBlock: [ : a : b | a value < b value ])

For more advanced software engineering queries have a look to the cool Moose ecosystem

by Hernán (noreply@blogger.com) at October 25, 2018 02:17 AM

October 24, 2018

Hernan Morales

Pharo Script of the Day: SPARQL access to DBPedia


Let's face it, how many times you could have a mix of Natalie Portman with Smalltalk code? :) If you install a little SPARQL wrapper library in Pharo, you could for example access the Natalie's movie list querying DBPedia by writing something like the following code in the SPARQL query language:


DBPediaSearch new
setJsonFormat;
timeout: 5000;
query: 'PREFIX dbpedia-owl: <http://dbpedia.org/ontology/>
SELECT DISTINCT ?filmName WHERE {
?film foaf:name ?filmName .
?film dbpedia-owl:starring ?actress .
?actress foaf:name ?name.
FILTER(contains(?name, "Natalie"))
FILTER(contains(?name, "Portman"))
}';
execute

To actually get only the titles you can use NeoJSON to parse the results:

((((NeoJSONReader fromString: jsonResults) at: #results) at: #bindings) collect: [ : entry | entry at: #filmName ]) 
collect: [ : movie | movie at: #value ]

And this is how results looks like:


by Hernán (noreply@blogger.com) at October 24, 2018 12:11 AM