1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
|
# showall no folding of any entries
#+STARTUP: showall
# showeverything show even drawer contents
# +STARTUP: showeverything
* Changes
:PROPERTIES:
:CUSTOM_ID: _changes
:END:
** update as of July 2021:
+ /Pire/ compiles again with ghc 8.8.4
+ with some tests passing (utests, filetests),
+ =_doctests.hs= is reduced for now, as show-prettyprint is not installable currently,
(upon which some tests did depend) and it's runing from the command line
(=cabal v2-exec doctest -- ...=)
+ other tests yet to be looked into (testsuite, roundtrip)
+ /indentation-trifecta/ used from its git source: https://github.com/adamsmd/indentation,
added as a /cabal-submodule/, and used with =--allow-newer=,
to make up for its version constraints as found on hackage
+ /MonadFail/ constraints added as necessary
+ use /cabal v2-*/ commands throughout
+ some work is underway of changing the infrastructure:
- rely on /Gauche(-Scheme)/, and /redo/ for the basics (I use [[http://news.dieweltistgarnichtso.net/bin/redo-sh.html][redo-sh]])
- and on /Haskell/, /Shake/ only once a basic environment is set up.
(=build.sh= -> =build.scm=, =cabalfile.hs= -> =cabal.scm=, for etc.),
+ docs yet to be updated
** December 2017
some changes towards ghc 8.8.4
complete rewrite as of December 2017,
summary of major changes and some design decisions
(ordered by topic), recent and older ones:
- Pire is based on Edward Kmett's Bound library instead of Unbound, used in Pi-forall
- While an older version of Pire included Pi-forall as a library, the current one does not.
It was used for testing Pire's parser
against Pi-forall's original parser at the time, and will possibly be reintroduced again.
Thus functions to translate between the various syntax representations, like =untie=
have disappeared as well for now.
** concrete syntax and white space aware parsing, naming conventions
- For every abstract syntax expression (=If=, say), a white space aware version (=If_=)
was given in the past, covering not only the abstract syntax essentials, but also any white
in between. Necessarily these expressions (declarations, modules etc.) were complicated:
While =If= was defined as
#+BEGIN_SRC hs
If (Expr t a) (Expr t a) (Expr t a) (Annot t a)
#+END_SRC
eg., its white space aware counterpart =If_= was
#+BEGIN_SRC hs
If_ (Token 'IfTokTy t) (Expr t a) (Token 'ThenTokTy t) (Expr t a) (Token 'ElseTokTy t) (Expr t a) (Annot t a)
#+END_SRC
each of the token and subexpressions allowing for trailing white space.
=expr_= was the parser of these white space aware expressions (and still is), =expr= being
the ordinary abstract syntax tree parser.
This naming convention applies to module names as well: =Modules_= eg. is the
white space aware version of =Modules=, with some exceptions: =WsAwareParser= vs. =Parser= notably.
These white space aware expression (like =If_= above) turned out
- likewise nopos (no position) operates on Absy_Bound (but on
ConcreteSyntax_Bound and Absy_Unbound as well with the help of type
classes): get rid of position information to make the parsers output
more human readable
** testing
- tasty/hunit test available in Pire/Tests.hs and Pire/FileTests.hs
There are currently 340 tests in Pire/Tests.hs and 46 tests in
FileTests.hs to provide compatibility with Pi-forall
- doctests in most of the source files, run them with
#+BEGIN_EXAMPLE
$ doctest -isrc src/Pire/Parser/Basic.hs
$ doctest -isrc Pire.Parser.Basic
$ doctest -isrc src/Pire/Parser/*
#+END_EXAMPLE
eg.
- The unit tests have been moved from the src directory to their own
tests directory, and are treated as a (cabal) test-suite utests, use
them eg. with:
#+BEGIN_EXAMPLE
$ cabal configure --enable-tests
$ cabal repl utests
#+END_EXAMPLE
#+BEGIN_EXAMPLE
>>> :l tests/Tests.hs
>>> main' "*"
>>> :l tests/FileTests.hs
>>> main' "*"
#+END_EXAMPLE
- testing has been improved: the doctests, while in place for a while
already, usage like this:
#+BEGIN_SRC bash
$ find ./src -iname "*.hs" | egrep -v -i "keepme|whatever" | xargs doctest -isrc
#+END_SRC
can now be called from a separate =doctests.hs= program in the tests
subdir, and are part of the the cabal test-suite, they can be called
togehter with:
#+BEGIN_EXAMPLE
$ cabal configure --enable-tests
$ cabal build
$ cabal test
#+END_EXAMPLE
Three tests currently: unit tests (utests), file tests (filetests),
and doctests. OK, more docs to come, how to use them, but these should
go in the docs section, not the changes section.
- a kind of literate programming for the docs introduced (with special
=// doctest= ... =// /doctest= comments, more details in the
[[https://github.com/reuleaux/pire/blob/master/docs/README.adoc#doctest_snippets][docs/README]],
ie. the other way around as extracting code snippets, as recently
introduced), the idea is to have readable adoc files, with examples
doctested nevertheless, cf. eg. the
[[file:parsec.xml#parsec_parsers][Parsec chapter]]
- The individual tests in Tests.hs pointing to precise
features/requirements each, were easier to handle in case of failures,
than the relatively complex file tests in FileTests.hs.
Thus more individual tests were added: there are 468 of them now (plus
the 57 file tests).
** t2s is handy when working on the command line
- Working on the command line it seems most natural to work with
Strings, like so
#+BEGIN_SRC hs
>>> V "a" :@ V "b"
...
>>> lam "x" $ V "a" :@ V "b"
...
>>> Ws_ (V "b") $ Ws "{-foo-}"
...
#+END_SRC
These are examples of String expressions
#+BEGIN_SRC hs
>>> :t Ws_ (V "b") $ Ws "{-foo-}"
Ws_ (V "b") $ Ws "{-foo-}" :: Expr [Char] [Char]
#+END_SRC
but the parser yields Text expressions, eg.
#+BEGIN_SRC hs
>>> nopos $ parse lambda " \\ x . a"
Lam "x" (Scope (V (F (V "a"))))
>>> :t nopos $ parse lambda " \\ x . a"
nopos $ parse lambda " \\ x . a" :: Pire.Syntax.Ex.Ex
#+END_SRC
#+BEGIN_SRC hs
>>> t2s $ nopos $ parse lambda " \\ x . a"
Lam "x" (Scope (V (F (V "a"))))
>>> :t t2s $ nopos $ parse lambda " \\ x . a"
t2s $ nopos $ parse lambda " \\ x . a" :: Expr String String
#+END_SRC
Yes, one /can/ use =:set -XOverloadedStrings= on the ghci (cabal repl)
command prompt, but this only so good: often enough, it's not clear
what is meant by ="foo"= then: the String, the Text?
- This t2s conversion one wants to do with the least effort of course,
and cheating at the /ermine/ source code (as for many things /Bound/
related as well), /bitraverse/ (/Bitraversable/ etc) comes in handy:
once expressed, how to bitraverse the expression tree in general, the
String to Text conversion of Expr t a becomes just a bitraversal with
T.unpack for both: its t and a parameters.
** position decoration...
not currently implemented...
- can use bitraversal to decorate the syntax tree with (very fine
grained) position information: ie. for every identifier, token, white
space etc, all this after parsing. (independent of the (Pos ...) info
wrapped around some exprs by the parser: didn't want to completely
change the parser again, and found this position info used too
sparingly)
had to change tokens for this to work: every token keeps track of the
string (text) parsed, in order to participate in this decoration with
position info
likewise had to reintroduce white space aware constructors for Type_,
TyBool_ etc, had removed them just recently (oh well!), because I
thought I could save them with Ws_ (Type t) Ws eg., but now we need
not only the white space after the Type, but also the string (text)
"Type" itself, thus Type_ t (Ws t), ie. the t in the middle.
anyway, happy with the position decoration, watch out for bound names,
though (some examples in Decorate.hs). This will hopefully not be a
problem as we need to open binders (instantiate lambdas etc), as we
navigate the tree.
beginning navigation in the concrete syntax tree (parsed w/ white
space), more to come, likewise more explanations, examples.
- /uwrap/ introduced in addition to /wrap/. /unwrap/ can be simpler
(rely on hoistScope ie.), recall: wrap allows bound vars to be wrapped
w/ their textual representation, for they can contribute to the
calculation of the position info)
- further groundwork for navigation in the white space aware syntax
tree, ie. functions/classes/general approuch in =Decorate.hs= and
=Refactor.hs= refined, to be continued.
- position info, ranges, and navigation in the ws aware syntax tree, and
refactoring split into pieces and moved to Refactor/*.hs
- Text2String supports modules now
- can use bifoldMap/foldMap on the decorated syntax tree (with position
info ie.) to calculate ranges (similar in spirit to bitraversal),
range calculation thus simplified
- first steps of walking the (ws aware) tree by line column number
- managed to even traverse ConstructorNames (use them as an
applicative), by means of a poor man's traversal on sets. decorateM
thus returns a module now (the old tuple version is still available as
decorateM'), decorated modules can thus be treated as any other pieces
of the zipper
- /Decorated/ typeclass and instances introduced, and constraints for
/Range/s simplified that way.
** pretty printing
- made pretty printing work on T.Text only, to simplify matters, but
provided printing of String Exprs/Decls/Modules etc by means of
Text2String.
# hm... - maybe useful
# https://stackoverflow.com/questions/698562/disabling-underscore-to-subscript-in-emacs-org-mode-export/701201
# but escaping w/ foo\_bar seems not to work
** simple exprs
- white space aware versions of the SimpleExpr parsers introduced as
well: =simple_expr=, =simple_term= etc
# --------------------------------------------------
- the Expr constructor Pos renamed to Position, so Pos could be used for
the constructor of position info as calculated / decorated (formerly
LineColumn), just this one constructor now (ie. From, To, AnyPos
merged to Pos)
|