What is WebScraping

-

Web Scraping



Many websites maintain their own API when they decided to share the information with public like Google, Cymon.io , StackOverflow, Twitter and so on. Few API's are accessible by everyone while few are accessible using a special key given for every registered individuals. The data being provided by these API are in structured format and can be easily be captured by users at a single sight.

Not all the websites hosted in the world maintain their API's to share the information.

But what if you need data from a website that does not maintain any API?? Here comes the idea of WEB SCRAPING. A small piece of something, especially one that is left over after the greater part has been used is called the scrap.(Definition from Wiki)

When we extract the useful information from a WEBsite , such a process of extraction is called WEB SCRAPING. Usually the process includes the conversion of the unstructured data (hosted on website) into structured data (extracted useful data)

Python can be used for scraping the web. It includes the libraries like Urllib2(fetches the URL) , BeautifulSoup(Extracts the information required) and many

One can get the clear idea of above mentioned libraries from their documentation.

LINK: Urllib2 Docuumentation

LINK:BeautifulSoup

Comments

Post a Comment

Popular

Traversal In A Binary Tree - Tree -3

-
Image
Traversal In Binary Tree Traversing a binary tree means visiting each node of the tree exactly once. At each node : N : Visit the node L : Traverse its left subtree R : Traverse its right subtree These actions can be performed in any order... (3! = 6 different ways of ordering the three tasks.) Different orders: NRL NLR LNR LRN RNL RLN But we follow the convention which states left subtree to be traversed before right sub tree. Thus we are left with three orders: NLR , LNR and LRN The orders are named according the position of the node i.e., position of N in the order. pre : NLR in  : LNR post : LRN PRE ORDER TRAVERSAL: ( NLR ) N : Visit the node L : Traverse the left subtree R : Traverse the right subtree IN ORDER TRAVERSAL: ( LNR ) L : Traverse the left subtree N : Visit the node R : Traverse the right subtree ...
Read more

Pre Order Traversal In Binary Tree

-
Image
Pre Order Traversal In Binary Tree Its NLR ordered traversal. The order goes this way: N: visit the node L: Traverse the left sub tree. R: Traverse the right sub tree. Lets check with an example. Consider the following example. Visit the node 1 Traverse the left sub tree of node 1 Visit the node 2 Traverse the left sub tree of node 2 Visit the node 4 Traverse the right sub tree of node 2 Visit the node 5 Traverse the right sub tree of node 1 Visit the node 3 Traverse the left subtree of node 3 Visit the node 6 Traverse the right sub tree of node 3 Visit the node 7 On completing the traversal we get the pre order if we follow the bolded nodes. Pre Order: 1 , 2 , 4 , 5 , 3 , 6 , 7
Read more

Tree data structure - 2

-
Image
Trees - 2 Types: Strictly binary tree: Degree is every node either 0 or 2. Full Binary tree: Degree of every node is 2. Complete Binary Tree:A binary tree in which every level , exccept possibly the last , is completely filled and all nodes are as far left as possible. Array Representation of binary tree: Also called as sequential representation or linear representation We use 1D array Store the data with indexed 1 i.e., root node at Array[1](Even we can use 0 based index ) If a node is at index k in the array, its left child is at index 2k and right child at index 2k+1. If a node at index k then its parent is at index floor(k/2) The size of array needed to store a tree of height h is 2 h -1 Linked list representation : struct node { struct node *left; struct node *right; int data; } This structure can be used for the binary tree. Self referential structure where first and second...
Read more