Question

[Question 4, 8 points total, including 4.1 - 4.3] This question is about understanding data representation and input/output in Spark. 4.1 (3 points) Spark uses a string to represent a vector. For example, (8. [1,4,5), 10.871,1.0,0.4471) is one such string. Write out the full vector that the above string represents: your answer: 4.2 (2 points) Very briefly, what is the main advantage of using such a sting representation for a vector? your answer: 4.3 (3 points) When using sedumasimilarities call on a RDD, the results coming back is a collection of Matusixentes instances. One of the instance looks like the following, MatrixEntex (109,385,0.032109) very briefly, write out what does this entry mean. your answer:

Answer 1

1.

In Spark we've many forms of vectors to represent the string in

For this example it can be sparse(8, [1,4,5], [0.871, 1.0, 0.447]) it has a size of 8, [1,4,5] represent the indices, the rest represent the values which are non-zero

It means that the vector has

0.871 at 1st position

1.0 at 4th position

0.447 at 5th position

2.

It's helpful in computation and storage management. Such a string is used to provide clear reference.

3.

The constructor MatrixEntry represents ith row and jth column and it's corresponding value of that entry

109 is row

385 is column

0.032109 is the value/entry

row and column are of type long

value is of type double

Hence this represents to enter that particular value in a matrix which is distributed at ijth position.

comment for any clarifications!