Consider a probability measure supported by a regular geodesic ball in a manifold. For any p larger than or equal to 1 we define a stochastic algorithm which converges almost surely to the p-mean of the measure. Assuming furthermore that the functional to minimize is regular around the p-mean, we prove that a natural renormalization of the inhomogeneous Markov chain converges in law into an inhomogeneous diffusion process. In compact and nonconvex manifolds we prove concentration and uniqueness of median under suitable assumptions on the support of the measure.