A client asked me, "When will my team be done with this project?" This is probably the bazillionth time I've been asked that agile project management question in one way or another. I have never once been asked, "How hard will my team have to think to develop this project?" Clients, bosses, customers and stakeholders care about how long a project will take. They don't care about how hard we have to think to deliver the project, except to the extent that the need to think hard implies schedule or cost risk.
I mention this because I find too many teams who think that story points should be based on the complexity of the user story or feature rather than the effort to develop it. Such teams often re-label "story points" as "complexity points." I guess that sounds better. More sophisticated, perhaps. But it's wrong. Story points are not about the complexity of developing a feature; they are about the effort required to develop a feature.
In a class a few years back, I was given a wonderful example of this. Suppose a team consists of a little kid and a brain surgeon. Their product backlog includes two items: lick 1,000 stamps and perform a simple brain surgery -- snip and done. These items are chosen to presumably take the same amount of time. If you disagree, simply adjust the number of stamps in the example. Despite their vastly different complexities, the two items should be given the same number of story points -- each is expected to take the same amount of time.
This example also points out another aspect of agile estimating, which is that we assume, in general, the right person for the job will do the work. We do not assume the little kid will finish school, go to med school, do a seven-year residency and only then begin the brain surgery while we have a skilled surgeon sitting in a cubicle licking stamps. Of course reality intrudes and occasionally the "wrong" person for a job does the job, but that will rarely be as dramatic as in this example.
So, story points are about the effort involved. Feel free to adjust your estimate of effort based on things like risk and uncertainty, but point-based estimating is about the time the work will take. It's what our clients, bosses, customers and stakeholders care about.