A quick Google suggests what you have.

If the code you have quoted is verbatim what you have tried, seems like you need to extract the parentheses and possibly a single or double quote, depending on the source css. The example source you have given has a single quote.

select-before(select-after(//div/@style, "backgound-image: url("), ")") 

Should be (notice the extra ' relating to url('...url'))

select-before(select-after(//div/@style, "backgound-image: url('"), "')")

But I don't think that would cause xpath to fail... It would just extract the wrong value

Edit:
Further reading suggests xpath 1.0 does have limited functionalities. But, like you, can't find anything concrete.