Friday, November 23, 2018

Understanding Misclassifier


A common issue that has to be addressed during model building  is the understanding the output and churning it . One important understanding is required in the term 'misclassifier'.

It is also required to understand the following terms
True positive , false positive, true negative , false negative.

Depending upon the business / technical problem the important misclassifier  need to be identified and churned .

For instance,
False positive means predicting a negative result as a positive result.

In the below example  a sample fictitious data is used for understanding.
It has 32 variables and 1000 records. The data is scaled suitably to use it for analysis, which is a typical data preprocessing activity.
It is partitioned for testing and training. 80% -20 %

Two models were build- ANN and CRT.

How should we understand the results?







                     Figure1 : Model built- ANN and CRT using SPSS Modeler.






Figure2: Results from CART algorithm 


Figure3: Response from the Artificial and Neural Network. 


Based on the given output by the models the question that arises is which should be chosen ?
Considering the figure 2, the false positive 64 % and the figure 3 gives a result of 55.4 % which mean that the the former model is saying the result as positive which is higher than the ANN model.
What is the conclusion ?
If we are bothered on a model which should say the facts as such , then both of those are dangerous. Say for instance we are working on a hospital data for which treatment is given for  a symptom and the model predicts that the person has a disease. Isn't this dangerous ? Hence work on the  misclassifier and refine the model . 

How to start python / learning programming ?

So, you have researched , googled, and finally came to a conclusion that need python for enhancement in career, business, automation and so on.....But where and how to start.

With my experience in automation,predictive analytics and interest in software, I have touched few points,if you think useful pick those. I walked through all the steps mentioned here.
1.How to start learning python ? Which are the best resources- websites / e- books / institutes/ tutors are the questions that are frequently asked by people ..
 My one line answer is your passion, your interest ,money and the google.
Start from any mode, but start today.
All the text books/ contents/ courses of online can give you information or make you walk around.
In my opinion swimming cannot be taught online. Likewise is the programming.
 You will not have the same set of theoretical programming that you have learnt in labs/ classes in real applications.Therefore the basics can be learnt in 15- 20 hours of simple training- it can even be self taught. I repeat- the word ' PASSION'.
2.You start  a simple program yourself,with a problem statement- addition of two number/ multiplication - logical questions or of any kind - be a self starter- later you can magnify the  problem level and improve  your programming skill.
It needs lot of guts to say this ! The reason being most of the resources give problems and also  solution which hinders our way of thinking. Solve it yourself--- YOU WILL GAIN CONFIDENCE.

3. Time matters- More the time you devote for programming, more will be your ability.

4. There is no substitute for your creativity and problem solving ability- use that. If you have not got any programming exposure- that is the best thing needed- you can think in a different mode.
All you need is the logic and  not the syntax. I know the experts with lot of experience still google for the syntax.
5. The software's capability is infinite- it means the problem you think today might have been experienced by most others years ago. The other skill needed is ' GOOGLING' the solution for your syntax.
While  we study in primary school, we were exposed to the languages,alphabets, words and sentences. Can mugging up an answer for a question help us anywhere ? Similarly is the programming

Know the data types, syntax - for few  and start writing programs. That's all the one you need.
Joining in groups, answering  questions asked in stackoverflow, attending contests can definitely make you pro!!!