Regression can be used on categorical responses to estimate probabilities and to classify.

I predict you'll find this logistic regression example with R to be helpful for gleaning useful information from common binary classification problems.

The Fisher information matrix for the estimated parameters in a multiple logistic regression can be approximated by the augmented Hessian matrix of the moment-generating function for the covariates.