Question

Part 2: Preperation for ML algorithms %run part2-setup.py 7. Write the function prepare_data(dataset_dict), which takes a dataset dictionary and returns a tuple (X,y) that holds the datamatrix and the labels In [3] : .Both X and y are of type numpy arrays . If the output variable is numeric (np.issubdtype(y.dtype, np.number)) it should be kept as is, otherwise it should be discretized using the command pandas.factorize) (use default function parameters). .Categorical features (whose columns are specified in the configuration file) should be transformed into binary values using the command pandas.get_dummies() . Add all columns even in the case of binary values, that is, keep drop first-False · Run the command pandas, get-dummies column by column, that is, on senes rather than a DataFrame (the reason is that the function ignores numeric values when applied to an entire dataframe and results in a different order of columns) The order of columns/features in X should be the following: original numerical columns followed by binarized categorical features . In addition to returning the tuple (X,y), the function should add to dataset_dict the values of X and y using the keys 'X' and 'y In [4] def prepare_data(datasetdict): Prepare datafile for learning algorithms (converts categorical values and separates x,y) Parameters dataset_dict: dictionary holding information about the dataset Returns (X,y) -tuple holding the data matrix and output variables Note: there is no need to call the function load_data, which is executed prior to calling the function prepare_data pass ### YOUR CODE HERE

Answer 1

def prepare_data(dataset_dict):

from sklearn.feature_extraction import DictVectorizer as DV

""DictVectorizer expects data as a list of dictionaries, where each dictionary is a data row with column names as keys:" "

vectorizer = DV( sparse = False )

X = vectorizer.fit_transform( dataset_dict)

y = vectorizer.transform( dataset_dict)

return (X,y)