Ask coding questions

← Back to all posts
2
Web Scraping Repl.it
vedprad1 (461)

Web Scraping Repl.it

Has anyone successfuly web scraped repl.it to get the code? When I tried to scrape one of my programs, all I got was this message:
Not sure what to do? Run some examples (start typing to dismiss)

All help appreciated!

Commentshotnewtop
1
PowerCoder (2)

You can get the html of replit using xpath

from lxml import html
from request import get

replit_page = get('https://repl.it/talk/ask?order=new') #I used replit's ask page
print(replit_page.text) #This will print out the html

replit_html = html.fromstring(replit_page.content)
print(replit_html.xpath('/html/body/div[1]/div/div[4]/div[3]/div/div[1]/h1/text()'))
#That will give you, "Online Music Hackathon," text

Heres a tutorial on how to web scrape with xpath in python https://docs.python-guide.org/scenarios/scrape/
Hope this helped (:

2
vedprad1 (461)

@PowerCoder : I meant in node.js. Sorry for the unclarity.

1
mat1 (2258)

This isn't exactly web scraping, but I made a small program 2 days ago that does this https://repl.it/@mat1/repl-test
Please mark this comment as the answer if it helped. Thanks!

1
ash15khng (216)

I don't think anyone has ever tried, and I don't think repl.it was made for getting code information from web scraping.

1
Geocube101 (190)

The most I've done is make a program to get the number of cycles a user has