Lightweight template-based parser build system. Simple prototyping. Comfortable debugging. Effective developing

by Flutter Awesome

parser_builder

Lightweight template-based parser build system. Simple prototyping. Comfortable debugging. Effective developing.

Version: 0.1.0

Early release version (not all built-in common buildres are implemented but can be used without them).
It is under development, but you can already play around. An example of a working JSON parser is included.

First advantage

  • Very simple and clear build and code generation system
  • Templates-based (visually intuitive) definition of parser builders
  • Parser declaration based on combinations of parsers
  • Support for generating parsers into functions or into statements (inlined parser code)
  • The performance of the generated parsers is quite high
  • The generated source code of the parsers is human-friendly
  • Very handy debugging and setting breakpoints
  • An elegant way to implement your own tracing which can easily be turned off
  • Very flexible error handling system (with support for nested errors)
  • Built-in error preprocessing procedures (grouping and flattening errors)
  • Fully customizable (according to your needs) error reporting procedures
  • Includes high-performance, most common built-in parser builders

Second advantage

Another advantage is that the build system is very simple and straightforward. Any programmer will understand it without much difficulty.
This means that an ordinary programmer can keep this software up to date.
You do not need to be a man of seven spans in the forehead for this.
The author of this software is also not such and is a simple person.
It’s just a simple but handy thing.
Use it and don’t worry about it stopping working.
This software is already so simple that it couldn’t be easier.

Documentation comming soon.
To get started, you can view and run the following scripts:

Generated simple JSON parser.
https://github.com/mezoni/parser_builder/blob/master/example/example.dart

Usage of generated JSON parser.
https://github.com/mezoni/parser_builder/blob/master/example/test_example.dart

A tool to create a simple JSON parser.
https://github.com/mezoni/parser_builder/blob/master/tool/build_example.dart

Included parser builders

built-in:

  • Named
  • Ref

branch:

  • Alt

bytes:

  • SkipWhile0 (not fully tested)
  • Tag
  • TakeWhile
  • TakeWhileMN

character:

  • Char
  • NoneOf
  • OneOf
  • Satisfy (not fully tested)

conbinator:

  • Eof (not fully tested)
  • MapRes (not fully tested)
  • Opt (not tested yet)
  • Value (not tested yet)

error:

  • Labeled (not fully tested)
  • Malformed (not fully tested)
  • Nested (not implenented yet)

multi:

  • FoldMany0 (not implenented yet)
  • FoldMany1 (not implenented yet)
  • Many0 (not implenented yet)
  • Many0Count (not implenented yet)
  • Many1 (not implenented yet)
  • Many1Count (not tested yet)
  • ManyMN (not implenented yet)
  • ManyTill (not implenented yet)
  • SeparatedList0 (not fully tested)
  • SeparatedList1 (not implenented yet)

sequence:

  • Delimited
  • Pair
  • Preceded
  • SeparatedPair (not tested yet)
  • Terminated

Example of generated code

Result<List<dynamic>>? _array(State<String> state) {
  Result<List<dynamic>>? $0;
  for (;;) {
    final pos = state.pos;
    Result<String>? $1;
    $1 = _openBracket(state);
    if ($1 != null) {
      Result<List<dynamic>>? $2;
      $2 = _values(state);
      if ($2 != null) {
        Result<String>? $3;
        $3 = _closeBracket(state);
        if ($3 != null) {
          $0 = Result($2.value);
          break;
        }
      }
    }
    state.pos = pos;
    break;
  }
  return $0;
}

Inlining code example

Eof parser has been inlined.

Result<dynamic>? _json(State<String> state) {
  final source = state.source;
  Result<dynamic>? $0;
  for (;;) {
    final pos = state.pos;
    Result<dynamic>? $1;
    $1 = _ws(state);
    if ($1 != null) {
      Result<dynamic>? $2;
      $2 = _value(state);
      if ($2 != null) {
        Result<dynamic>? $3;
        for (;;) {
          if (state.pos >= source.length) {
            $3 = const Result(null);
            break;
          }
          state.error = Err.expectedEof(state.pos);
          break;
        }
        if ($3 != null) {
          $0 = Result($2.value);
          break;
        }
      }
    }
    state.pos = pos;
    break;
  }
  return $0;
}

Example of error reporting

Unhandled exception:
FormatException:
line 1, column 1: Expected: [, {, ", string, number, false, null, true
  ╷
1 │ 123.`
  │ ^
  ╵
line 1, column 1: Malformed number
  ╷
1 │ 123.`
  │ ^^^^^
  ╵
line 1, column 5: Unexpected: '`'
  ╷
1 │ 123.`
  │     ^
  ╵
#0      main
#1      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:295:32)
#2      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

Performance

Current performance of the generated JSON parser.

Parse 10 times: E:\prj\test_json\bin\data\canada.json
Dart SDK JSON   : k: 1.00, 45.87 MB/s, 468.00 ms (57.69%),
Simple JSON     : k: 1.73, 26.46 MB/s, 811.20 ms (100.00%),

Parse 10 times: E:\prj\test_json\bin\data\citm_catalog.json
Dart SDK JSON   : k: 1.00, 81.21 MB/s, 202.80 ms (59.09%),
Simple JSON     : k: 1.69, 47.99 MB/s, 343.20 ms (100.00%),

Parse 10 times: E:\prj\test_json\bin\data\twitter.json
Dart SDK JSON   : k: 1.00, 49.60 MB/s, 109.20 ms (77.78%),
Simple JSON     : k: 1.29, 38.58 MB/s, 140.40 ms (100.00%),

OS: Microsoft Windows 7 Ultimate 6.1.7601
Kernel: Windows_NT 6.1.7601
Processor (4 core) Intel(R) Core(TM) i5-3450 CPU @ 3.10GHz

The data for the test is taken from here:
https://github.com/serde-rs/json-benchmark/tree/master/data

More info…

To be continued…

GitHub

View Github