How To Make A Language: Parsing

The last tutorial didn't really give an in depth explanation as to what everything was doing aside from some comments, however for parsing I feel I should explain how it works.

Markdown was really weird on this post.

We are going to be using a recursive descent parser, written all by hand and with no dependencies.

Lets start with an expression `8 + 5 * 4 / 2`

The first thing we are going to do is tokenize it, so this expression will tokenize to this

``````1: ("Number", 8)
2: ("Operator", "+")
3: ("Number", 5)
4: ("Operator", "*")
5: ("Number", 4)
6: ("Operator", "/")
7: ("Number", 2)``````

Im going to do a little experiment here, and if it works you should be able to read the above stream of tokens just like an rdp parser. When you see `Add(), Sub(), Mul(), Div() and Literal()` in the text below jump to the corresponding label!

Left = Sub()

``````Is the next token a + operator? If so then:
Right = Sub()
Return (Left, "+", Right)

Otherwise return the left side``````

Sub:
Left = Mul()

``````Is the next token a - operator? If so then:
Right = Mul()
Return (Left, "-", Right)

Otherwise return the left side``````

Mul:
Left = Div()

``````Is the next token a * operator? If so then:
Right = Div()
Return (Left, "*", Right)

Otherwise return the left side``````

Div:
Left = Literal()

``````Is the next token a / operator? If so then:
Right = Literal()
Return (Left, "/", Right)

Otherwise return the left side``````

Literal:
Return the current token and advance to the next one

Now, if you went through everything above and kept track of where you were in the sequence you will notice that you end with a tree that looks roughly like this:

You are viewing a single comment. View All
[deleted]

# I like this

Soz about the 10yr old making Python classes rather than a programming language

LoganSpong (52)

@BXRStudios I get it already! I took down the post!

[deleted]

@LoganSpong I don’t think you are @CSharpIsGud but if you are hello otherwise I was talking to @CSharpIsGud

LoganSpong (52)

@BXRStudios No, I'm the 10 year old that made the uhh... thing.

CodeLongAndPros (1537)

Can people just

# Get over it!

It is not a big deal! Point it out once, then be done!

You can ask them to change the name, but really?

This kind of behavior is downright mean. Some people just like to make stuff and show off what they made!

## </rant>

CSharpIsGud (798)

@CodeLongAndPros Mean is kind of relative, since people will call calmly pointing out that importing a module doesn't make a new language mean while also ones that say the same thing in an insulting tone mean so there are 2 kinds of mean on replit

CodeLongAndPros (1537)

@CSharpIsGud There are some people that don’t like it because it is “not canonical”. I mean, sure, you can point it out, but don’t fixate on it. I’ve pointed some things to that effect, but I don’t focus on them. You could say that some people are stating a fact, and others are fixated on that same fact.

[deleted]

Well let's just stop here then and all just take a break. :)

bramley (248)

Please check the Repl.it TOS, @LoganSpong

LoganSpong (52)

@bramley I did. I could'nt find the section at which you're talking about. Which one is it?

LoganSpong (52)

@bramley Is it the copyright one? Like this: You may be held accountable for damages (including costs and attorneys' fees) for misrepresentation or bad-faith claims on the infringement of any User Content found on and/or through the Service on your copyright.

Or is it this one?
If we learn of any user that is under the age of 13, we will immediately terminate the user’s Account and delete any and all information we may have about that user.