ipernity API development / Discussions

Problem with a limit: How to test your application? Of course, good programmers build a caching system, but until this point you need to do a lot of calls to get data to “play” with while coding your application.

--
Coming from a group home page (?)

Caching can solve some problems for sure. The PHP API kit I used to play with Flickr API had caching already integrated and you had to have a MySQL db to use it.
But when you intrinsically haver to do with great quantities of data, caching doesn't help (combined with the 100 items per page limit which forces you to do more roundtrips to obtain the same quantity of data). Making analysis on a single day worth of shots, I already seen the data changing while I was looping... Perhaps the solution is in caching/proxying on the server instead of on the client side of the XML-RPC interaction.

--
Coming from a group home page (?)

my API-implementation will be able to do result-caching ;-)
but i need just a little bit more time before i release it ... ;-)

I think there is a misunderstanding with the 3000 items limit. We are limiting the search.doc responses pages to the first 3000 elements. Ex: if a search returns 200 000 results and you have per_page=10 you will be able to browse 300 pages max.
The API calls limits are much higher (100K request/day or so).

There isn't a misunderstanding, Christophe. I understood it well and I'd like to know the reason, if it isn't to reduce the server load.
Some example:
- a month worth of public uploads: 150.000 shots
- Ojisanjake stream: 12000+ shots
- bigger groups: 10000+ shots
These are three situations where we'd have to do an extra external loop, changing some parameters, to go through it.
If you want to make a bot to help admins to manage group pools, if you want to sum the total visits on Jake's stream, if you want to find the more viewed shots of September, ... all examples of situations where this 3000 items limit add complexity to code.
So if the reason for the limit is to reduce server load, well, I can understand it, but I think it isn't effective: I can code around it: it's better to introduce a limit of calls per hour to force us to streamline the code (but possibly this testing phase isn't the right moment for it...).
You have to find a balance between server costs and quality of service for developers: I think that the quantity of Flickr mashups available is one of the greatest reasons for their success --> give the developers the possibility to grow your success ;-)
--
Coming from a group home page (?)

well, I think it's also meant to reduce server-load or to prevent abuse!
Searching is in general is quite a complex task, and will increase load nevertheless.
To get an unlimited list of user docs, you could use album.getList, the same thing for groups would be nice (and possibly will come when group-functions of the API will be published) ... somehing like container.getList ...

so, the limit of search results shouldn't be a problem and will prevent abuse of the function. ;-)

I have to read more of the documentation, but, after your reply, I still miss the reason for limiting that API, considering that there are so many ways to go around the limit ;-)

--
Coming from a group home page (?)

it's for sure not about "getting around" a certain limit ;-)
the point I wanted to say is, that it's (generally) much more "expensive" to search with a certain amount of constraints (user_id, tags, etc.) than asking for "get all docs of a user" or "get all docs of a group".
And since searching is that expensive, it has a limit (meant: "don't let users use this function for all purposes"). If every api user would use the search-function in that way, ipernity would be possibly overloaded!

Exactly beatmaster! read my response to Roberto.

Hello Roberto, the search method should only be used to search ;) And browsing over X thousand of results in a response list should not makes sense most of the time. This limit is also here to prevent abuse & server load.
There are and will be more methods to browse the complete list of documents from a member docstream, album, group, tags, etc... So do not worry about this we will not put some stupid limits there ;)
But I understand that actually the doc.search is the only way to browse a user doctream or a group. We will add the needed methods next week.
Anyway, we will listen to all your needs. Let's wait a week or two and speak about limits again ;)

Thanks Christophe.
My only intention is to help you as I can and I wanted to understand what was the ratio about this limitation.
I'm sure you'll extend the API and all will be simpler for developers.
(I must confess: I haven't read the documentation about each API; I'm too lazy and there were no (working) links back to the methods list ;-) )

--
Coming from a group home page (?)

Thank you Roberto.