OpenSSL is an open source library that implements the SSL and TLS protocols. More than that, OpenSSL also provides an exhaustive general cryptography library, with both high- and lowlevel API's.
EVP is an OpenSSL API that provides a high-level interface to cryptographic functions. While OpenSSL also has direct interfaces for operations like signing data with an RSA key, the EVP library separates the operations from the actual backend used. That way, the actual implementation that is used can be changed, and one can specify an engine to use for the operations.
One of the engines that can be selected in recent versions is a pkcs11 engine.
What does this imply? If your application uses the EVP library, it's very easy to let your users use their HSM for their cryptographic needs, as long as their HSM is supported by a driver.
For an example of usage, see getting things to work 4.1.
If you use this API, it's still a lot like you'd have used the old low-level functions, and you can still use your specific internal cryptographic algorithms. However, a very simple addition makes the code a lot more flexible:
Instead of NULL you can provide a filename, but with null the value of the environment variable is used. In the provided file, the user can add possible engines. For a better user experience, it is nice to provide a configuration or command-line option that specifies the file, instead of letting the user provide it through an environment variable.