Question

I want to use BING API for web searching using python. Search result should only contain PDF. I added the advanced operator (filetype: pdf) in the search query. But seems it not working.

******** Python code **************

import requests

def bing_search(query):
    url = 'https://api.cognitive.microsoft.com/bing/v5.0'
    # query string parameters  
    payload = {'q': query, 'filetype':'pdf','responseFilter':'Webpages'}  
    # custom headers
    headers = {'Ocp-Apim-Subscription-Key': '9126a2280100424b90b85d764a18dc34'}
    # make GET request
    r = requests.get(url, params=payload, headers=headers)
    # get JSON response
    return r.json()

j = bing_search('Machine Learning')
print(j.get('webPages', {}).get('value', {}))

I could not figure it how to use advanced operator (filetype:pdf) to filter search results.

Could anyone please suggestion me how to use it?.

Thanks

Answer 1

Your code is good. There are only 2 edits needed and it worked. The changes are in bold.

see the code below:

##################################################

import requests

def bing_search(query):
    url = 'https://api.cognitive.microsoft.com/bing/v5.0/search'
    # query string parameters
    payload = {'q': query}
    # custom headers
    headers = {'Ocp-Apim-Subscription-Key': '9126a2280100424b90b85d764a18dc34'}
    # make GET request
    r = requests.get(url, params=payload, headers=headers)
    # get JSON response
    return r.json()

j = bing_search('Machine Learning&filetype=pdf')
print(j.get('webPages', {}).get('value', {}))

##################################################

This gives me the following output: (as you will see it worked and only contain pdf)

[{'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.0', 'name': 'Introducing Machine Learning - MathWorks', 'url': 'https://www.mathworks.com/content/dam/mathworks/tag-team/Objects/i/88174_92991v00_machine_learning_section1_ebook.pdf', 'displayUrl': 'https://www.mathworks.com/content/dam/mathworks/tag-team/Objects/i/...', 'snippet': 'Introducing Machine Learning. Machine learning teaches computers to do what comes naturally to humans and animals: learn from experience. Machine learning algorithms use computational methods to “learn” information directly from data without relying on a predetermined equation as a model. The algorithms adaptively improve their performance as the number of samples available for learning increases. What is Machine Learning? 3 ntroducing Machine Learning More Data, More Questions, Better ...', 'dateLastCrawled': '2019-03-25T13:51:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.1', 'name': 'Introduction to Machine Learning (pdf) - Alex Smola', 'url': 'http://alex.smola.org/drafts/thebook.pdf', 'displayUrl': 'alex.smola.org/drafts/thebook.pdf', 'snippet': 'Introduction to Machine Learning (pdf) - Alex Smola', 'dateLastCrawled': '2019-03-25T02:46:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.2', 'name': 'Introduction to Machine Learning Using Python', 'url': 'https://in.pycon.org/2011/static/files/talks/11/Introduction_To_ML_Partial_2.pdf', 'displayUrl': 'https://in.pycon.org/2011/static/files/talks/11/Introduction_To_ML...', 'snippet': "Definition: A computer program is said to 'learn' from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by", 'dateLastCrawled': '2019-03-25T15:22:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.3', 'name': 'Understanding Machine Learning: From Theory to Algorithms', 'url': 'https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/understanding-machine-learning-theory-algorithms.pdf', 'about': [{'name': 'Understanding Machine Learning: From Theory to Algorithms'}], 'displayUrl': 'https://www.cs.huji.ac.il/~shais/UnderstandingMachineLearning/...', 'snippet': 'Understanding Machine Learning Machine learning is one of the fastest growing areas of computer science, with far-reaching applications. The aim of this textbook is to introduce', 'dateLastCrawled': '2019-03-26T22:21:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.4', 'name': 'INTRODUCTION MACHINE LEARNING - Stanford AI Lab', 'url': 'http://ai.stanford.edu/~nilsson/MLBOOK.pdf', 'displayUrl': 'ai.stanford.edu/~nilsson/MLBOOK.pdf', 'snippet': 'Machine learning methods can be used for on-the-job improvement of existing machine designs. The amount of knowledge available about certain tasks might be too large', 'dateLastCrawled': '2019-03-27T11:39:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.5', 'name': 'Download - A Course in Machine Learning - CIML', 'url': 'http://ciml.info/dl/v0_8/ciml-v0_8-all.pdf', 'displayUrl': 'ciml.info/dl/v0_8/ciml-v0_8-all.pdf', 'snippet': '10 a course in machine learning ated on the test data. The machine learning algorithm has succeeded if its performance on the test data is high. 1.2 SomeCanonicalLearningProblems There are a large number of typical inductive learning problems. The primary difference between them is in what type of thing they’re trying to predict. Here are some examples: Regression: trying to predict a real value. For instance, predict the value of a stock tomorrow given its past performance. Or predict ...', 'dateLastCrawled': '2019-03-26T14:32:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.6', 'name': 'The Discipline of Machine Learning – Tom Mitchell', 'url': 'http://www.cs.cmu.edu/~tom/pubs/MachineLearning.pdf', 'about': [{'name': 'Machine learning'}], 'displayUrl': 'www.cs.cmu.edu/~tom/pubs/MachineLearning.pdf', 'snippet': 'Machine Learning is a natural outgrowth of the intersection of Computer Science and Statistics. We We might say the defining question of Computer Science is “How can we build machines that solve problems,', 'dateLastCrawled': '2019-03-24T23:16:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.7', 'name': 'Machine Learning 1.1 - Cognizant', 'url': 'https://www.cognizant.com/de-de/pdf/Machine_Learning.pdf', 'displayUrl': 'https://www.cognizant.com/de-de/pdf/Machine_Learning.pdf', 'snippet': 'Machine Learning labels a new wave of enterprise software which is able to learn from data and make predictions, explanations, detect anomalies, make', 'dateLastCrawled': '2019-03-24T14:37:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.8', 'name': '1 What is Machine Learning?', 'url': 'https://www.cs.princeton.edu/courses/archive/spr08/cos511/scribe_notes/0204.pdf', 'about': [{'name': 'Machine learning'}], 'displayUrl': 'https://www.cs.princeton.edu/courses/archive/spr08/cos511/scribe...', 'snippet': 'The machine learning paradigm can be viewed as “programming by example.” Often we have a specific task in mind, such as spam filtering. But rather than program the computer to solve the task directly, in machine learning, we seek methods by which the computer will come up with its own program based on examples that we provide. Machine learning is a core subarea of artificial intelligence. It is very unlikely that we will be able to build any kind of intelligent system capable of any ...', 'dateLastCrawled': '2019-03-13T19:08:00'}, {'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.9', 'name': 'A Few Useful Things to Know about Machine Learning', 'url': 'https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf', 'displayUrl': 'https://homes.cs.washington.edu/~pedrod/papers/cacm12.pdf', 'snippet': 'A Few Useful Things to Know about Machine Learning Pedro Domingos Department of Computer Science and Engineering University of Washington Seattle, WA 98195-2350, U.S.A.', 'dateLastCrawled': '2019-03-27T13:24:00'}]