420chan now has a web-based IRC client available, right here
Leave these fields empty (spam trap):
Name
You can leave this blank to post anonymously, or you can create a Tripcode by using the float Name#Password
Comment
[*]Italic Text[/*]
[**]Bold Text[/**]
[~]Taimapedia Article[/~]
[%]Spoiler Text[/%]
>Highlight/Quote Text
[pre]Preformatted & Monospace text[/pre]
1. Numbered lists become ordered lists
* Bulleted lists become unordered lists
File

Sandwich


Community Updates

420chan now supports HTTPS! If you find any issues, you may report them in this thread
Simple Linux Dev by Clara Woshtug - Mon, 20 Feb 2017 03:11:37 EST ID:YL0YFsWq No.36519 Ignore Report Quick Reply
File: 1487578297992.jpg -(130895B / 127.83KB, 1080x1080) Thumbnail displayed, click image for full size. 130895
I want so accomplish something very simple with my CentOS webserver, but I'm not sure where to begin.

I need to visit a URL to some JSON then convert the JSON to a SQL insert statement, and I want it to run every 30 seconds lets say, forever.

There's nothing there that sounds particularly difficult but I don't even know what search terms I should be looking at. Any help is appreciated.
>>
Charles Sonkinmat - Mon, 20 Feb 2017 05:26:21 EST ID:dluvNLbx No.36520 Ignore Report Quick Reply
>There's nothing there that sounds particularly difficult

I don't know about difficult, but none of it made sense to me.
>>
Clara Woshtug - Mon, 20 Feb 2017 05:54:45 EST ID:YL0YFsWq No.36521 Ignore Report Quick Reply
>>36520
Every 30 seconds;
  1. download the content at http://site.com/thing.json
  2. extract information from the json objects and insert the info into a local database

I've got curl and wget installed now, so I can 'get' the file, and I have the vixie-cron cronjob service installed, so all I need now is a way of parsing the json data and squirting into the DB.
>>
Charles Sonkinmat - Mon, 20 Feb 2017 12:41:13 EST ID:dluvNLbx No.36522 Ignore Report Quick Reply
>>36521
If you're dead set on doing this the unix way, you want jq. Nothing is easier or cleaner than Python though.
>>
Matilda Duttingnane - Mon, 20 Feb 2017 13:05:04 EST ID:WLOo3E7i No.36523 Ignore Report Quick Reply
>>36521
create a cron job that runs a bash script every 30 seconds
the script will look something like this if you are using a mySQL database:
wget -o result.txt http://site.com/thing.json
data=$(cat result.txt)
mysql -uroot -ppassword -e "USE mydb; INSERT INTO mytable VALUES ('$data');"
>>
Clara Woshtug - Mon, 20 Feb 2017 17:16:44 EST ID:YL0YFsWq No.36524 Ignore Report Quick Reply
>>36522
Cool, jq looks like exactly what I needed, thanks.
I was going to hack something together but this looks perfect
https://stedolan.github.io/jq/tutorial/

>>36523
Thanks, seems like I'm on the right track. I've saved your example for tinkering with.
>>
Phineas Barryway - Tue, 21 Feb 2017 22:17:07 EST ID:YL0YFsWq No.36525 Ignore Report Quick Reply
I literally just spent 8 hours trying to install and set up fail2ban because, of course, within about 30 seconds of the server coming online it was being hammered by chinese bots. Turns out it was a problem with fail2ban that is being fixed right now. Fuck.
>>
Fanny Gudgespear - Fri, 24 Feb 2017 16:04:55 EST ID:9QSfnS0r No.36570 Ignore Report Quick Reply
>>36525
Ugh, fail2ban is a stupid stopgap measure and I can't stand that it's a common practice.
I it were up to me everything that's not intended to be accessed by the public would be behind a ssh tunnel. I know end user authentication and all that stuff is still an issue.
Again if it were up to me I'd just fucking force them to use a password which has enough entropy. Ever had to beg to be white-listed from a spam blacklist because some retard use the name of there dog as a password?
Cuz fail2ban won't save you that hassle, believe me.
>>
John Hocklehood - Sat, 25 Feb 2017 00:04:50 EST ID:YL0YFsWq No.36571 Ignore Report Quick Reply
>>36570
Fair enough, this is the first time I've had to administer a Linux server or even any kind of server. I noticed there were a lot of failures in the log so fail2ban was what I ended up with after a few minutes of searching around.

There's only me accessing it and it's just a cheap $5 box that I can trash and rebuild whenever but I'll look into what you're suggesting if you give me a link or two to readup.
>>
Frederick Fannerforth - Tue, 28 Feb 2017 07:02:30 EST ID:YL0YFsWq No.36575 Ignore Report Quick Reply
>>36571
Not that anyone cares but the server is up and running and everything is ticking along nicely.
I have scripts that are recording all the threads created on another chan board so I can do some analysis on trends, repost, etc.

Once the infrastructure was in place it wasn't hard to add some logging for 420chan. I am polling http://api.420chan.org/stats.json every 30 mins so hopefully in the future that I can produce some dank graphs, but there's isn't much info exposed.


Report Post
Reason
Note
Please be descriptive with report notes,
this helps staff resolve issues quicker.