What would you if this was your friend benefits?

met a guy when i was 18 we went out for a few months he ended it and we became fwb for a year i was fine with that,I then met somebody else and moved away for 4 years he asked me 2 meet him before i went i didnt but over that 4 years we stayed in touch and he came up to meet me I didnt do anything with him ,3 years ago i returned home with a baby and we became fwb for two years then last year we would see each other for a couple months have a few dates i would meet his friends then he gets scared i think like were 2 close or something and goes away and then comes back again,when we are togather he would stroke my hand and hold me all night I realized about a year ago that I love him and want more but iam scared of rejection but i allso iknow that he always comes back do you think he is useing me i think 8 years is a long time and there must be more to it ,.please help iam so confused