The broad field of neuromorphic computing presents itself as an attractive alternative to tackle the issues currently faced by conventional implementations of neural networks (NNs) in various artificial intelligence (AI) applications, in particular their heavy energy, resource, compute and data hungry operation and training, which could be addressed more efficiently. In this context, this work presents a mixed signal, hardware-based and configurable design of a low power spiking neuron, as well as a memristor-based synapse design with the potential to allow the sidestepping of these problems, by enabling system-level implementations of NNs that approach the learning and operation processes from a computational neuroscience inspired angle. This design takes inspiration from the predictive coding (PC) [1] theory of brain function. In PC, the global dynamics of a neural network evolve in such a way that a global free energy minimum, understood as the maximization of the correspondence between the internal model held by the neural network and the sensory data that it receives, can be reached by having neurons and synapses modify their behaviour based only on their locally available information. This work showcases the preliminary steps before a larger hardware-based system level implementation of that variety of NN can be carried on, by designing the constituent blocks of such a system, mainly neurons that possess internal dynamics with the aim of aligning their real activity with their expected activity via an error-minimizing feedback loop, and memristive-based synapses that exploit the read-disturb phenomena observed in these devices [2] to autonomously update their conductances based on the activities of the neurons that they are linking. Simulation results will be presented along their corresponding transistor-level schematics and the theoretical background behind this approach.