Holy crap this is longer than I intended.
Pitching
Win/Losses/Saves
Absolutely have zero value. If Scott Kazmir is on the mound, goes five, gives up five runs, and has his team bomb the opposing starter for seven, he's in line for a win. If James Shields pitches eight, gives up a lone run, and the opposing starter shuts the Rays down, Shields might get charged with a loss. Troy Percival can walk the bases loaded, allow two sac flies, and as long as he kept the lead, get credited with a save. W/L/S do not speak for a players actions, but rather his team actions. Starters on good teams will probably have more wins than starters on poor teams, the same goes for saves.
ERA
The most common pitching stat around. We use ERAs scale (runs allowed per nine innings) for other run average metrics like tRA and FIP, but ERA itself isn't forth telling about pitcher value. Luck, like poor defense or a shoddy non-error call, can inflate ERAs. Plus, pitchers don't control hit rates nearly as much as people think, which is also why WHIP is pretty meaningless. ERA+ is equally uninformative.
Strikeouts
Are good, very, very good. Swinging strikeouts are the best since they imply the pitcher has plus stuff or velocity and are seemingly more sustainable than strikeouts looking - which relies on an off hitter and a potentially favorable strikezone. Strikeouts usually translate well from year to year and minors to majors.
Walks
Are bad, very, very bad. For high strikeout pitchers they're more acceptable, given the ball will be put into play less, but for average to below average strikeout types, they can be killer.
LD/GB/FB
Or line drives, groundballs, and flyballs. If you're ranking them in terms of what you'd prefer: GBs >> FBs >>>> LDs. Liners are the batted ball type most likely to go for a hit. Pitchers who give up a ton of line drives simply don't make it to the majors, or if they do, they have a freaking warehouse of rabbits feet. Flyballs can result in homeruns, the more flyballs, the more homeruns (depending on the park). Yeah, groundballs can go for hits, but how often is a grounder going for extra bases?
FIP
Fielding Independent Pitching. Okay, here's where the whole "isolating the pitcher from the team" process gets started. The pitcher can control three things: walks, strikeouts, and homeruns allowed. Hence, FIP takes each of those into account and puts it on a scale so that league average FIP is equal to league average ERA. FIP+ is the "compared to league average" form.
tRA
Graham's stat is FIP on hGH. The pitchers are charged not only for homeruns, walks, and strikeouts, but also the batted ball type. So pitchers with a ton of line drives allowed are going to have a higher tRA than pitchers with mostly groundballs. tRA* is the regressed form, and tRA+ is the "compared to league average" form.
BABIP
This plays hand and hand with LD/GB/FB. Most pitchers have batting averages on balls in play between .290 and.310. Anything on the other end of the extremes, without something peculiar involved with the LD rate, is probably going to regress back to that range. Are there some outliers? Absolutely. Good/bad defense can affect this number heavily. A better point of reference than "Batting Average Against", but neither tells you much about the pitchers talent. The best method for a quick BABIP estimator is an average over the last three years.
Hitting
Batting Average
Only tells one side of the story. A player can hit .310, but if his OBP and SLG are .320/.400, he's not quite as useful as you would assume.
On-Base Percentage/Average
Tells a slightly larger side of the story. It's usually best to combine OBP with walk rates and or an iso discipline stat (which is to say OBP - BA).
Slugging Percentage
Again, tells one part of the story, the bases per at-bat side.
wOBA
Don't let the name confuse you, this isn't just OBA/OBP under a different name. Instead it's a Tom Tango created metric that uses linear weights (aka: weighs how much each event contributes to a run) assigns a value to, and then spits out in a form that we can easily turn into runs created. FanGraphs version takes into account SB and CS as well.
wRAA
FanGraphs version of wOBA runs created above league average. The "Value" section of a player page includes a park adjusted version of this metric.
BABIP
See above.
LD/GB/FB
See above.
RBI
Oh heavens to Betsy I hate RBIs. People will look at a player's runs batted in and assign labels like "run producer" to him. Perfect example: Jose Guillen. Look, the guy hits for some power, but he's not a very good hitter, especially not for a corner outfielder. In 2008 Guillen had 97 RBIs, yet had a wRAA of -5. Yeah, a below average hitter had nearly 100 RBIs.
Homeruns
They're nice, but they aren't the only way to score. Also, a player who hits 30 homeruns and does nothing else is not valuable, but a player who hits 5 homeruns and does other stuff can be.
Stolen Bases
Are nice. Just make sure you're succeeding ~70% of the time.
Defense
Errors
Too opinion based, plus if a player doesn't have the range to get to a ball, how can it be an error?
Fielding Percentage
Consider the above. If Jason Bartlett gets to a ball deep in the hole, then throws the ball away, that's an error. If Derek Jeter takes two steps, then lets the ball shoot through the gap while blowing bubbles, that's not an error. You see the problem here.
UZR
FanGraphs defensive stat of choice. Developed by MGL. Takes the park into equation and adjusts based on such.
CHONE
A defensive system developed by Rally/Sean Smith.
PMR
A defensive system developed by David Pinto.
Plus/Minus
A defensive system developed by John Dewan and The Fielding Bible group. Provided in PLAYS form, so multiple that by 0.8 (the value of turning a single into an out) and you'll get the runs form. These are formed by a team of experts watching each play and giving a "+" or a "-" based on the play made. If Bartlett makes a play that another shortstop missed, he gets a "+", if Bartlett misses a play another shortstop made, he gets a "-".
Use these together, average them, and you'll have a nice idea of how valuable (or how not valuable) a player is.
Defensive spectrum:
DH | 1B | LF | RF | 3B | CF | 2B | SS | CA
The left side is the easier positions, the right the harder.
Win Values
Positional adjustments:
CA - 22.5 runs.
1B/COF - -7.5 runs
2B/CF/3B - 2.5 runs
SS - 7.5 runs
DH - -22.5 runs, but since batters are usually ~ -5 runs worse as a DH/pinch hitter, we give them a five run bonus, so only penalize the player -17.5 runs.
Replacement adjustments:
This is a fancy way of saying: a replacement level player is ~20-22.5 runs worse than an average player. Think Ray Olmedo.
Replacement player
Average player: replacement player :: superstar player : average player
Batting
Park adjusted wRAA.
Fielding
UZR totals.
Value Runs
The total runs the player contributed.
Value Wins
Value Runs/10 (or the amount of runs for a "win")
pLI
Leverage, aka how tough or untough a situation is for a pitcher. 1 is the standard for starting pitchers, 1.8-2 for closers, 1.3-1.5 for set-up men, ect, and 0.5 or so for mop-up men.
WPA
Win probability added, usually found next to the win expectancy chart, seen below.
Projection systems
PECOTA
The best of any projection system. This is a Baseball Prospectus feature.
ZiPS
Found at BaseballThinkFactory. Pretty good.
CHONE
Found at baseballprojection.com. Again, pretty good.
Marcels
The most basic of all projection systems, yet still does pretty decent considering it's basically the last three years of data, an age modifier, and a ton of regression. All other projection systems should aim to outperform Marcels or otherwise quit.
Analysts to trust
Keith Law
Rob Neyer
Dave Cameron
Tom Tango
MGL
Sky Kalkman
Peter Bendix
Graham MacAree
Matthew Carruth
Jeff Sullivan
Eric Seidman
Anyone at BTB or FanGraphs
Miscellaneous
A great defender CAN be as valuable as a great hitter can.
Position DOES matter when evaluating offense.
Pitchers do not follow the same aging/prime curve as hitters.
Things like vet presence, leadership, great chemistry have marginal effects on performance.
Most hitters are better at home than the road.
One year of offensive data = three years of defensive data.
Consider each data point on its own. So, if Player A has UZRs of 5, 0, -5, that doesn't mean he's on the decline. Use a three year average (and add weight to the most recent results if you wish) instead.