if you have any questions i'll be glad to answer

also, there's a lot of resources online, you can just google it and you'll certainly find an answer, probably from stackoverflow

That's inspiring

I will go for it

unless you're writing something like a preprocessor, it's absolutely not necessary for parsing

i am

i see

are you using a state machine

and the source is in a seperate string, not a mmapped file or similar

are you using a state machine

i’m not sure

state machines traverse through the input one character at a time

that is what i’m doing

and whenever it finds a valuable character, it changes its state, which allows it to expect different kind of tokens

then yes i am

so you have a state/context class instance which holds all of the valuable information, right

then yes i am

hmm

how are you doing recursiveness in a state machine

i mean, for a parser its understandable because of the nested nodes

but for a state machine i dont find the need to use recursiveness

the way my parser currently works is as a parser combinator eg, Char, String, Any, Sequence, Or, OneOrMore, ZeroOrMore, Range, Stack, and so on it also does basic backtracking and input modification

are you writing a parser for a language

if you like, this is my Rules.h for the parser https://gist.github.com/abba506dd2083e4909f0e522f4981c41

what is the usage of this

i mean, what does it do (your project)

it’s current usage is as a preprocessor

for what kind of input

cpp?

more or less this is what i currently have https://gist.github.com/788d44482d202d0a1388f08d86b1842f

oh my god you use new so many times it kinda unnerves me

but i see what you're doing

by any chance does it work in top of patterns?

or is it a drop-in replacement of the original preprocessor that the compiler has

i’m not sure what you mean

does it work like a compiler preprocessor

also do you have a general repo for all of these different files

does it work like a compiler preprocessor

at the moment it works similar to gcc -E but without all the #Line output

i see

i found the repo in your github

https://github.com/mgood7123/CPP

i see

very cool

if you want, the is the current tests (output) i am running for it https://gist.github.com/mgood7123/db1150767b6065521c432962ca61d02d (most of the tests are in different files, eg the iterator tests are in Iterator.h, Rule tests are in General.cpp (because IDE doesnt like #ifdef GTEST_API_ being in Rules.h for some reason), basic iterator match tests are in IteratorMatcher.h, CPP tests (a few, not a lot) are in General.h )

if you want, the is the current tests (output) i am running for it https://gist.github.com/mgood7123/db1150767b6065521c432962ca61d02d (most of the tests are in different files, eg the iterator tests are in Iterator.h, Rule tests are in General.cpp (because IDE doesnt like #ifdef GTEST_API_ being in Rules.h for some reason), basic iterator match tests are in IteratorMatcher.h, CPP tests (a few, not a lot) are in General.h )

the code is overly complicated but the result is very good

have you considered following the factory pattern

i think it would make your code a little cleaner

for example, you could make a factory for rules and then derive different kind of rules from it

some (very important) parsers, such as the one for V8 engine, use this pattern

i’m currently experementing with a stack based rule https://gist.github.com/2ae9b3090f80a472d0df0f702466aba9 as i am trying to implement https://gist.github.com/mgood7123/168c4b17dd814fd977a2db88af5d9f5f in order to correctly recursively expand function macro’s

for example, you could make a factory for rules and then derive different kind of rules from it

how would i do that?

how would i do that?

there are some examples online: https://gist.github.com/sacko87/3359911

but i do not understand this enough to be able to implement it https://gist.github.com/mgood7123/168c4b17dd814fd977a2db88af5d9f5f

there are some examples online: https://gist.github.com/sacko87/3359911

looks complicated

very verbose explanation, but basically it means: traverse all characters after the #define ... read the first parameter if you encounter a ( token, this means the first parameter is a function macro set the context to parse function-like arguments, generate a function macro rule, insert function parameters to the rule context continue reading and save the rest of the macro arguments into an arguments vector save the rule object into a stack

looks complicated

for me, using class inheritence and overloading is much easier to implement struct OneOrMore : RuleHolder { OneOrMore(Rule * rule, Action action = NO_ACTION) : RuleHolder(rule, action) {} using Rule::match; virtual IteratorMatcher::MatchData match(Iterator<std::string> &iterator, bool doAction = true) override { IteratorMatcher::MatchData match = rule->match(iterator, doAction); if (match) { while (true) { IteratorMatcher::MatchData tmp = rule->match(iterator, doAction); if (!tmp) break; match.end = tmp.end; match.matches += tmp.matches; } if (doAction) action(Input(iterator, match, match.matches)); } return match; } };

if you do that you end with verbose dependencies to other classes

if you do that you end with verbose dependencies to other classes

how so?

very verbose explanation, but basically it means: traverse all characters after the #define ... read the first parameter if you encounter a ( token, this means the first parameter is a function macro set the context to parse function-like arguments, generate a function macro rule, insert function parameters to the rule context continue reading and save the rest of the macro arguments into an arguments vector save the rule object into a stack

for the define or for the statements AFTER the define ? eg, for define x(a) a or for x(bar)

how so?

you're forced to initialize a RuleHolder within your derived Rule class, which makes you depend of the RuleHolder object for accessing properties of your class. Imagine you wanted to access some property x, which both your current class and RuleHolder has. Under this context, you'd need to forcefully use this-> to specify that you want to access this object property (aka, accessing members of dependent base classes)

you're forced to initialize a RuleHolder within your derived Rule class, which makes you depend of the RuleHolder object for accessing properties of your class. Imagine you wanted to access some property x, which both your current class and RuleHolder has. Under this context, you'd need to forcefully use this-> to specify that you want to access this object property (aka, accessing members of dependent base classes)

https://isocpp.org/wiki/faq/templates#nondependent-name-lookup-types

you're forced to initialize a RuleHolder within your derived Rule class, which makes you depend of the RuleHolder object for accessing properties of your class. Imagine you wanted to access some property x, which both your current class and RuleHolder has. Under this context, you'd need to forcefully use this-> to specify that you want to access this object property (aka, accessing members of dependent base classes)

yes

for the define or for the statements AFTER the define ? eg, for define x(a) a or for x(bar)

the define token itself should set a state so the parser knows you're now parsing arguments for a definition macro

but i meant the parameters for the definition macro

another one is struct Char : Rule { char character; Char(char character, Action action = NO_ACTION) : character(character), Rule(action) {} using Rule::match; virtual IteratorMatcher::MatchData match(Iterator<std::string> &iterator, bool doAction = true) override { auto match = IteratorMatcher::match(iterator, character); if (match && doAction) { action(Input(iterator, match, match.matches)); } return match; } };

whats char for

when i read your code, i think of Rule as a class that defines the macro itself, rather than individual segments of it

RuleHolder is basically a RAII-like class for holding mulitple Rule instances so the same rule instance can be used in multiple rule declerations for example auto x = new Char(‘x’); auto seq1 = new Sequence({x, x}); // match xx auto or1 = new Or({ seq1, x, new Any() }); // match xx, x, otherwise anything except EOF

when i read your code, i think of Rule as a class that defines the macro itself, rather than individual segments of it

Rule is the base class which basically tells the user that this class is a Rule object and as such can be added to a collection of other Rule objects without having to worry about managing rules of different object types eg trying to add a Foo object to a collection of Bar objects cannot be done unless Foo extends Bar, or unless you use std::pair or similar which would then get very complicated as you do not know which rules you may encounter eg someone may create a new type of rule and then your class is broken because it does not accept this new type of rule because it does not extend Rule

ahhhhhhhhhhhhhhhhhhhhh

i see now

i did something like this before

but with wildcards, and rules didnt have unique classes, instead they set a state and as long as the string continued matching, it would be compliant with the pattern

but with wildcards, and rules didnt have unique classes, instead they set a state and as long as the string continued matching, it would be compliant with the pattern

https://github.com/coalio/rfscss/blob/master/src/wildcard.cpp

cool

also, why do you use new so much

is there a special advantage in it rather than allocating all of these objects in the stack

do you use them in many other scopes, why dont you use smart pointers if so

because class inheritence does not work without new eg Rule obj = Char(‘x’); a.match(“x”); // calls Rule::Match instead of Char::Match

and referneces would just complicate the grammar (eg every rule would have to be pre-defined before use and you cannot just construct the grammar on the fly)

i see

have you considered using templates instead

so a.match calls Rule::Match which calls DerivedRule::match