path: root/changes.org

# showall          no folding of any entries
#+STARTUP: showall


# showeverything   show even drawer contents
#  +STARTUP: showeverything


* Changes
  :PROPERTIES:
  :CUSTOM_ID: _changes
  :END:




** update as of July 2021: 
+ /Pire/ compiles again with ghc 8.8.4
+ with some tests passing (utests, filetests), 
+ =_doctests.hs= is reduced for now, as show-prettyprint is not installable currently, 
  (upon which some tests did depend) and it's runing from the command line 
  (=cabal v2-exec doctest -- ...=)
+ other tests yet to be looked into (testsuite, roundtrip)  
+ /indentation-trifecta/ used from its git source: https://github.com/adamsmd/indentation,
  added as a /cabal-submodule/, and used with =--allow-newer=,
  to make up for its version constraints as found on hackage
+ /MonadFail/ constraints added as necessary
+ use /cabal v2-*/ commands throughout
+ some work is underway of changing the infrastructure: 
  - rely on /Gauche(-Scheme)/, and /redo/ for the basics (I use [[http://news.dieweltistgarnichtso.net/bin/redo-sh.html][redo-sh]])
  - and on /Haskell/, /Shake/ only once a basic environment is set up.
    (=build.sh= -> =build.scm=, =cabalfile.hs= -> =cabal.scm=, for etc.),
+ docs yet to be updated




** December 2017
   some changes towards ghc 8.8.4


complete rewrite as of December 2017,
summary of major changes and some design decisions 
(ordered by topic), recent and older ones:


- Pire is based on Edward Kmett's Bound library instead of Unbound, used in Pi-forall

- While an older version of Pire included Pi-forall as a library, the current one does not.

  It was used for testing Pire's parser 
  against Pi-forall's original parser at the time, and will possibly be reintroduced again.  

  Thus functions to translate between the various syntax representations, like =untie=
  have disappeared as well for now.



** concrete syntax and white space aware parsing, naming conventions

- For every abstract syntax expression (=If=, say), a white space aware version (=If_=)
  was given in the past, covering not only the abstract syntax essentials, but also any white
  in between. Necessarily these expressions (declarations, modules etc.) were complicated:

  While =If= was defined as

  #+BEGIN_SRC hs
    If (Expr t a) (Expr t a) (Expr t a) (Annot t a)
  #+END_SRC
 
  eg., its white space aware counterpart =If_= was

  #+BEGIN_SRC hs
    If_ (Token 'IfTokTy t) (Expr t a) (Token 'ThenTokTy t) (Expr t a) (Token 'ElseTokTy t) (Expr t a) (Annot t a)
  #+END_SRC

  each of the token and subexpressions allowing for trailing white space.

  =expr_= was the parser of these white space aware expressions (and still is), =expr= being
  the ordinary abstract syntax tree parser.

  This naming convention applies to module names as well: =Modules_= eg. is the
  white space aware version of =Modules=, with some exceptions: =WsAwareParser= vs. =Parser= notably.


  These white space aware expression (like =If_= above) turned out 



- likewise nopos (no position) operates on Absy_Bound (but on
  ConcreteSyntax_Bound and Absy_Unbound as well with the help of type
  classes): get rid of position information to make the parsers output
  more human readable




** testing


- tasty/hunit test available in Pire/Tests.hs and Pire/FileTests.hs
  There are currently 340 tests in Pire/Tests.hs and 46 tests in
  FileTests.hs to provide compatibility with Pi-forall


- doctests in most of the source files, run them with

  #+BEGIN_EXAMPLE
      $ doctest -isrc src/Pire/Parser/Basic.hs
      $ doctest -isrc Pire.Parser.Basic
      $ doctest -isrc src/Pire/Parser/*
  #+END_EXAMPLE

  eg.


- The unit tests have been moved from the src directory to their own
  tests directory, and are treated as a (cabal) test-suite utests, use
  them eg. with:

  #+BEGIN_EXAMPLE
      $ cabal configure --enable-tests
      $ cabal repl utests
  #+END_EXAMPLE

  #+BEGIN_EXAMPLE
      >>> :l tests/Tests.hs
      >>> main' "*"
      >>> :l tests/FileTests.hs
      >>> main' "*"
  #+END_EXAMPLE


- testing has been improved: the doctests, while in place for a while
  already, usage like this:

  #+BEGIN_SRC bash
      $ find ./src -iname "*.hs" | egrep -v -i "keepme|whatever" | xargs doctest -isrc
  #+END_SRC



  can now be called from a separate =doctests.hs= program in the tests
  subdir, and are part of the the cabal test-suite, they can be called
  togehter with:

  #+BEGIN_EXAMPLE
      $ cabal configure --enable-tests
      $ cabal build
      $ cabal test
  #+END_EXAMPLE

  Three tests currently: unit tests (utests), file tests (filetests),
  and doctests. OK, more docs to come, how to use them, but these should
  go in the docs section, not the changes section.


- a kind of literate programming for the docs introduced (with special
  =// doctest= ...​ =// /doctest= comments, more details in the
  [[https://github.com/reuleaux/pire/blob/master/docs/README.adoc#doctest_snippets][docs/README]],
  ie. the other way around as extracting code snippets, as recently
  introduced), the idea is to have readable adoc files, with examples
  doctested nevertheless, cf. eg. the
  [[file:parsec.xml#parsec_parsers][Parsec chapter]]


- The individual tests in Tests.hs pointing to precise
  features/requirements each, were easier to handle in case of failures,
  than the relatively complex file tests in FileTests.hs.

  Thus more individual tests were added: there are 468 of them now (plus
  the 57 file tests).





** t2s is handy when working on the command line


- Working on the command line it seems most natural to work with
  Strings, like so

  #+BEGIN_SRC hs
      >>> V "a" :@ V "b"
      ...
      >>> lam "x" $ V "a" :@ V "b"
      ...
      >>> Ws_ (V "b") $ Ws "{-foo-}"
      ...
  #+END_SRC



  These are examples of String expressions

  #+BEGIN_SRC hs
      >>> :t Ws_ (V "b") $ Ws "{-foo-}"
      Ws_ (V "b") $ Ws "{-foo-}" :: Expr [Char] [Char]
  #+END_SRC



  but the parser yields Text expressions, eg.


  #+BEGIN_SRC hs
      >>> nopos $ parse lambda " \\ x . a"
      Lam "x" (Scope (V (F (V "a"))))
      >>> :t nopos $ parse lambda " \\ x . a"
      nopos $ parse lambda " \\ x . a" :: Pire.Syntax.Ex.Ex
  #+END_SRC




  #+BEGIN_SRC hs
      >>> t2s $ nopos $ parse lambda " \\ x . a"
      Lam "x" (Scope (V (F (V "a"))))
      >>> :t t2s $ nopos $ parse lambda " \\ x . a"
      t2s $ nopos $ parse lambda " \\ x . a" :: Expr String String
  #+END_SRC

  Yes, one /can/ use =:set -XOverloadedStrings= on the ghci (cabal repl)
  command prompt, but this only so good: often enough, it's not clear
  what is meant by ="foo"= then: the String, the Text?



- This t2s conversion one wants to do with the least effort of course,
  and cheating at the /ermine/ source code (as for many things /Bound/
  related as well), /bitraverse/ (/Bitraversable/ etc) comes in handy:
  once expressed, how to bitraverse the expression tree in general, the
  String to Text conversion of Expr t a becomes just a bitraversal with
  T.unpack for both: its t and a parameters.


** position decoration...
   not currently implemented...


- can use bitraversal to decorate the syntax tree with (very fine
  grained) position information: ie. for every identifier, token, white
  space etc, all this after parsing. (independent of the (Pos ...​) info
  wrapped around some exprs by the parser: didn't want to completely
  change the parser again, and found this position info used too
  sparingly)

  had to change tokens for this to work: every token keeps track of the
  string (text) parsed, in order to participate in this decoration with
  position info

  likewise had to reintroduce white space aware constructors for Type_,
  TyBool_ etc, had removed them just recently (oh well!), because I
  thought I could save them with Ws_ (Type t) Ws eg., but now we need
  not only the white space after the Type, but also the string (text)
  "Type" itself, thus Type_ t (Ws t), ie. the t in the middle.

  anyway, happy with the position decoration, watch out for bound names,
  though (some examples in Decorate.hs). This will hopefully not be a
  problem as we need to open binders (instantiate lambdas etc), as we
  navigate the tree.

  beginning navigation in the concrete syntax tree (parsed w/ white
  space), more to come, likewise more explanations, examples.


- /uwrap/ introduced in addition to /wrap/. /unwrap/ can be simpler
  (rely on hoistScope ie.), recall: wrap allows bound vars to be wrapped
  w/ their textual representation, for they can contribute to the
  calculation of the position info)

- further groundwork for navigation in the white space aware syntax
  tree, ie. functions/classes/general approuch in =Decorate.hs= and
  =Refactor.hs= refined, to be continued.

- position info, ranges, and navigation in the ws aware syntax tree, and
  refactoring split into pieces and moved to Refactor/*.hs

- Text2String supports modules now

- can use bifoldMap/foldMap on the decorated syntax tree (with position
  info ie.) to calculate ranges (similar in spirit to bitraversal),
  range calculation thus simplified



- first steps of walking the (ws aware) tree by line column number



- managed to even traverse ConstructorNames (use them as an
  applicative), by means of a poor man's traversal on sets. decorateM
  thus returns a module now (the old tuple version is still available as
  decorateM'), decorated modules can thus be treated as any other pieces
  of the zipper

- /Decorated/ typeclass and instances introduced, and constraints for
  /Range/s simplified that way.




** pretty printing

- made pretty printing work on T.Text only, to simplify matters, but
  provided printing of String Exprs/Decls/Modules etc by means of
  Text2String.


# hm... - maybe useful
# https://stackoverflow.com/questions/698562/disabling-underscore-to-subscript-in-emacs-org-mode-export/701201
# but escaping w/ foo\_bar seems not to work

** simple exprs


- white space aware versions of the SimpleExpr parsers introduced as
  well: =simple_expr=, =simple_term= etc




# --------------------------------------------------


- the Expr constructor Pos renamed to Position, so Pos could be used for
  the constructor of position info as calculated / decorated (formerly
  LineColumn), just this one constructor now (ie. From, To, AnyPos
  merged to Pos)