JEE questions are a joy to solve if one is not under time-pressure. While not insolvable, all problems require certain "tricks" and a person who has just memorized the subject matter will most probably end up getting them all wrong. Unlike GRE General Tests which have become controversial, JEE measures one's mastery over the subject areas covered and thus is supposed to be culturally unbiased.
Unlike ETS which regularly conducts reseach to determine the validity of its tests (GRE, GMAT, SAT, ...) combined with factors like GPA and subject tests to act as a predictor of success in school, I am not aware of any research done to validate JEE with academic success at IITs. If you know of any, please inform me.
Here I will state my findings based on informal observations at IIT Bombay. In short, I was amazed at how well GPAs correlated with JEE ranks. In the EE department, Narendra Karmarkar was #1 on JEE, Gurjar was #2, with Shah, Iyer, and others following them closely. After 5 years, their GPAs were in almost same order! Karmarkar got the gold-medal, and Gurjar was behind him by a single grade. Exceptions were there, but my guess is that correlation between the JEE ranks and the IIT graduating GPAs was very high, perhaps in the range of 0.50 or better. It is extremely remarkable considering that we are comparing a very narrow range of people - people who had scored in the top 1-2% of the applicants courageous enough to attempt the JEE.
If you are wondering about my problem with the JEE, then I have to say that it is too difficult. If the goal is to find a Ramanujan or even a dozen Ramanujans, perhaps a test like JEE is best for that. If, however, if the goal is to find 1000 or so good applicants to educate in IITs, JEE appears to be a bad tool. Moreover, for the applicants who took the JEE but were denied admission to IITs, JEE in general, is a morale-crusher.
Surely more than a dozen. Many more. At least in the range of tens of thousands each year if it has to emerge as a major industrial power. Goal of JEE amd Indian education system should be to optimize the development of engineers it needs. Unfortunately IITs and JEE don't do that. All they do is to separate out the cream of the crop, about a 100 or so, who are left outdoors to be immediately gobbled up by the vultures, err I meant the USA. Rest are left to fend for themselves.
There is a common misconception among IITians that since they scored in the top 1-2% of the applicants, their scores must be quite close to each other. Reality is quite different. This discovery arose from my own experience.
When I took the JEE, mathematics section was on the first day. During that year, it had two separate sections - regular and modern mathematics. One could attempt only one section out of these two. I started out with the modern mathematics section, after an hour or so I decided to switch to the regular section because the problems there looked more solvable, and after half an hour or later I was back to the modern mathematics section and so on. Finally when I reached home I told my mom that it is over and I won't get into IIT because I messed up my JEE. She told me not to worry and take the physics and chemistry sections the next day for formality. Physics was my favorite subject and everything went smoothly. I didn't like chemistry much but did fairly okay in that.
Now I was here at IIT with a JEE rank of about 250, wondering how I got in. One day I went to the admissions office and asked about my score on JEE and was directed to a posting on the wall. There hung all the total scores. Karmarkar had scores 271 out of 300, followed by Gurjar at 270. To my surprise, if my memory serves me right, a score of about 150 was good enough for rank around 500. Clearly there is a vast difference between the scores of #1 ranked person and say #1000 ranked person.
My theory is that since the test is so difficult that even the bright people who find the first one or two questons difficult just give up and panic and start getting things wrong. In a poor country like India where so much is riding on one test, to be extremely nervous is quite natural. JEE does not have any simple questions to build up confidence. Difficult problems keep arriving till it completely breaks you down. A few survive but for most positive scores are possible because of the partial credits only. I read that the recent passing cutoff score in the mathematics section was 17%!
Just like the grading scheme at IIT, JEE is meant to put you down. Except for a few dozen, rest get many answers wrong and the majority of the applicants didn't even get a single answer correct. Even if they join the IITs or in worse case they are rejected and have to study at other engineering colleges, confidence in one's abilities is gone. That is really sad.
A test like JEE is perhaps the only solution in India to assure people of fairness. The test could be designed like GRE Subject Test in Mathematics, Computer Science, Engineering or similar, where 80 or so questions have a few simple questions at the beginning, a few moderately difficult ones in the middle and a few - about 20 or so more difficult ones towards the end. It will allow most applicants to perform at their best. It will probably not distinguish between Karmarkar and Gurjar, but who cares. Admit the top 2000 or so, don't assign the departments to them. Review their performances at IIT, maybe have another test at the end of 2 years and based on those two factors and their interest, assign them their departments. Is this too much to ask for?