This paper focuses on an electro-hydraulic servo system, which is derived from a shaking table. It proposes a control scheme based on a back propagation (BP) neural network, whose weights are trained by the particle swarm optimization (PSO) according to the fitness, which is determined by the input and the feedback signals. Each particle of PSO includes weights and thresholds of BP. The movement of each particle is adjusted by its local best-known position and the global best-known position in the searching space. With the update, a satisfactory solution can be achieved. In order to show the performance of the proposed control scheme, the designed network is also trained and tested by BP only. The comparisons between the PSO-BP and BP networks demonstrate that the PSO-BP one has better performance than that of BP, both in convergence speed and in convergence accuracy.