Question

Web crawler guideline

Required development spec.

- Please use Python only

Recommend python library

- Scrapy or beautiful soup (Any others is ok!)

Please develop these two.

1. Basic crawler

a. Read the website by “URL”

b. Save the result (save mysql database)

c. Show the result via website

2. Scheduled crawler

a. Read the website by “URL” with scheduled time (ex, 1minute)

b. Save the result (save mysql database)

c. Show the result and crawling schedule status via website

d. Scheduler parameter: (repeat-time), (start from datetime),(until to datetime)

e. Can immediately stop the scheduler by using crawler id.

Answer 1

You Can simply Do the Basic Crawl On PHP

Here Function is used

In the Url, You Can Put Your Website
Initialize Database And COnnect Database Using Username, Password And Connect Required Database Name and Create Table on PHPMyAdmin.
Show Content On website Using Echo.