Question

I'm having a difficult time understanding how a tie is determined using k-NN. I had a homework problem and was able to calculate the distance for the data point being tested, however, I don't have a good understanding of what 'k' means. When determining the decision, there were 2 choices (high or low) but one of the k values was a tie & I am confused on how I figure that out. I maybe could understand if there were 2 or more of Euclidean distances that were equivalent to each other but that wasn't the case with the data set. Can anyone break this down in simplistic/dummy terms?

Answer 1

Answer :

KNN has no model but to store the entire dataset, so no learning is needed. Efficient implementations may store the data using complex data structures such as k-d trees to easily lookup and fit new trends during prediction. Since the entire training dataset is stored, you will want to think carefully about the accuracy of the training data. Curating it could be a good idea, reviewing it regularly as new data becomes available, and deleting inaccurate and obsolete data. In my opinion, the best way to break a tie for a nearest k neighbor will be to lower k by 1 before you break the tie. It will still function regardless of the voting weighting system, as a tie is unlikely if k = 1. If you were to raise k, you would not be able to guarantee a tie break pending your weighting scheme and number of categories. Suppose I can only select two people to inquire about Sue. I've selected her best mate, and then ... hmmm ... Who'd be the second? Mum, dad, uncle, and sister may have a relation. I might get very different perceptions of Sue depending on which of those people I chose.

A tie can occur when two or more points are equidistant from an unclassified observation, making it difficult to choose which neighbors. it is clear which two neighbors are to be chosen. you are accepting K and you ought an even number of classes Like 2, choosing a K value for an odd number is a good choice to prevent a tie. Or the opposite, if you have an odd number of groups, use an even number for K. Through extending K through 1 and looking to the class of the next most similar instance in the training dataset, links can be broken consistently. While the issue of neighbor selection will be solved by a three-nearest neighbor classification system, Besides, there is still the likelihood that selection relations exist on the larger collection of data irrespective of the value of k. To elude relations, unusual numbered k's are preferred. Nevertheless, tie scores in k-nn indicate low confidence in the prediction and I do not suggest that you rely on the result. In addressing the problem raised by the previous process, it is possible to weigh the neighbors so that those closest to the non-observed point have a "greater vote"This method will result in both L and N being listed as A because the neighbors A are closer to each other by comparison keeping this in mind, you should understand that there is no simple mandate about what to do when there is no definite winner in majority voting. You can use either an odd k at all times or some injective weighting. In the case of neighbors 3 to 5 sharing the same distance from the point of interest, you can either use only two, or use all five. KNN again is not some algorithm derived from complex mathematical analysis, but simply an intuition. Whether you want to deal with those individual situations is up to you.

**** Kindly LIKE the answer. It will help and motivate me. ****