[Most Recent Entries] [Calendar View] [Friends]
Below are the 50 most recent journal entries recorded in
LiveJournal Development's LiveJournal:
- lj-preformatted: true
- lj-opt_preformatted: true
- lj-preformatted: 1
- lj-opt_preformatted: 1
- A companion to synsuck which can consume PubSub content (where the payload is an Atom feed) into type Y journals. This would presumably take the form of a special Jabber client daemon which handles the subscribing and then makes sure that the recieved items end up in the right journals on the site.
- A component which publishes posted journal content from all journals. I'm still not sure of the best approach for this yet, but whatever happens it should only be called on to publish entries for which there are active subscriptions (or else it'll get really behind on LiveJournal.com). I'm currently a little confused about who is responsible for distributing the message to the various servers where there are subscribed clients, though. This component would serve a similar purpose to the /data/atom output, but without all the polling.
- Latest-RSS.bml - Although this is RDF, it is not properly formatted. Specifically, the fact that <lj:mood id='' /> uses an un-namespaced "id" as an attribute. Fixing this up is a one line patch, available here.
- RSS 1.0 feeds for entries - Although the forms we currently export (RSS 2.0 and Atom) can be converted to RDF (RSS 1.0 is an RDf-based format, compared to RSS 2.0, which is just XML), it would be nicer for use with RDF tools to provide an RDF form. This is (almost) trivially simple, and could allow for adding the Metadata described in this lj-dev post with almost no work.
- FOAF updates - There are a couple changes which could be made to FOAF which would be useful to many tools without significant additional data load for the site: the display of actual locations (using the VCARD spec), exporting birthdates in a more commonly used format (using the bio: namespace, rather than an RDF predicate which is not in common use) and adding a user picture as a depiction of the user are the most common requests I've seen. (The beginnings of a patch for this are available.
| Saturday, January 29th, 2005 | |
| 5:07 pm [dblockdotorg] |
backdated entries with ATOM API It doesn't seem like it's possible to post backdated entries with ATOM API (at least from reading the code). I am getting: The remote server returned an error: (500) Internal Server Error. Server Error - Unable to post new entry. Protocol error: Incorrect time value: Your most recent journal entry is dated 2004-12-16 11:09, but you're trying to post one at 2003-10-15 04:10 without the backdate option turned on. Please check your computer's clock. Or, if you really mean to post in the past, use the backdate option. My posted feed is of this format: entry generator title created content Did I miss something or can this be a feature request? Thx dB. |
| 12:55 pm [mpnolan] |
Multiple interest search Hi, I'm Mike. I wrote a comment statistics program a while ago, so some of you may love/hate me for that one. Now I've got another idea. I'm planning on writing a program that, given two interests X and Y, returns all users with both interests. This would be very easy to do if I had something like interestdata.bml only returning usernames with an interest rather than interests by username. Since nothing like this seems to currently exist, I wrote a small script named userinterests.bml. I scrapped it together from reading interestdata.bml and interests.bml, so I'm not sure if it works/how efficient it is. But I figured I'd contribute something rather than just requesting such a page be created. So what's the possibility of userinterests.bml being applied to the LJ source? |
| 12:06 pm [grahams] |
Blogger API Question I am writing a little python automation script for the Sunday Night Film Club to ease the effort required to announce the week's movie in several different forums (LJ, MT, Craigslist, Newsletter). I have been using the LJ Blogger API, and it's been working great, except I can't figure out how to turn on LJ's "preformatted" property for the event. I've read this document, and based on that I've tried adding the following text to the beginning of the post: Am I missing something, or is this prop not supported (or is there some completely seperate way of handling this property)? Thanks |
| Friday, January 28th, 2005 | |
| 12:10 am [cemcom] |
fdata.bml source Hi all, Where can I find the source code for the fdata.bml data stream? I've found the source for interestdata.bml, which seems somewhat similar, but fdata.bml (which seems like it should be in the same livejournal/htdocs/misc directory) isn't there. Any ideas? The reason I want this is because I want to try to my hand at writing a cdata.bml script (to return all the members of a community) which I could then maybe get added to the LJ codebase, under some restricted access if necessary. The reason for this and discussion are in this previous post. Thanks, Eugene |
| 2:22 am [mart] |
Answers I, for one, would like to see some of the promised answers to questions in the earlier post about ownership of code. (Of course, I'm most concerned about my comment, but some official input in the other big threads of discussion would be nice too.) |
| Saturday, January 22nd, 2005 | |
| 5:35 pm [fxool] |
Journals and WAP
I wanted to give a possibility to our server's users to read journals using WAP-access. So I
created S2 style, that outputs WML. Since It works on my journal. I tried it out by myself on other journals, specifying the "?s2id=..", to see how it looks, and it works, too. However, PHP cannot load others' journal pages with this style, because it is an anonymous user. Can there be something done, so this stuff could work? I don't care about protected entries,
it's OK not to get these. But can PHP be authenticated as a some registered user, using something
like this – "http://wapuser:pass@someLJpowered.server/u I'm a co-administrator of our LJ-powered Latvian blog server. I'm a "trusted" user, I can make
this layer / style public or whatever. I have access to the "bowels" of our server,
if that can be useful somehow. Kristaps / |
| Thursday, January 20th, 2005 | |
| 2:48 am [bradfitz] |
Power-loss post-mortem The post you've all been waiting for! Why we lost all our power, and why it took us so long to come back up afterwards.... (Warning: it's late and I'm tired/rambly, so this post might be incoherent... go ahead and ask questions...) Why we lost all our power... Another customer in the facility accidentally pressed the EPO button, then depressed it, replaced the protective case, and left the building. Intenap all thought it was their UPS systems failing, but then logged into them, saw EPO shutdown notifications, and couldn't find any EPO cases open or pressed, so probably freaked out for a bit thinking there was a short in the walls that triggered the EPO, only to get a confession a day or so later. EPO, by the way, stands for Emergency Power Off and it's a national fire/electrical requirement for firefighters to be able to press these big red buttons near all exits that turn off all power in the entire data center. This is the second time this has happened to us in the years I've been there. The first time the button was unlabeled and unprotected and some dude thought it opened the door. This time we have no clue why it was pressed... maybe that dude tripped and fell onto it... mystery. Internap will be putting alarms and tamper-proof indicators on the plastic cages surrounding the EPO buttons now, though, so at least if this happens again in the future they'll know why. Anyway, moving on.... Why it took us so long to come back up... Ton of reasons: Faulty mobos/NICs: We have 9 machines with faulty motherboards with embedded NICs that don't do auto-negotiation properly. They only work with certain switches, so they reboot fine, but then their gigabit network comes up at 100 half duplex or something that doesn't work. To get them back up they need somebody at the NOC to plug them into a compatible switch, let them autonego, then switch them to their real switch. Setting the speed/duplex settings on both the host and/or switch themselves doesn't work.... most annoying. We're getting working dual-port gigabit PCI NICs for those machines, rather than replacing their motherboards. Database start-up: All but a couple of our machines came back up when the power was restored, less than an hour later, but on our databases we intentionally don't have the database start back up on boot. In normal circumstances, if a single machine dies, it died for a reason and we want to investigate it. Normally that doesn't matter either, because we have 2+ of everything. But when every single database restarts, that leaves us with no alive databases, and we have to manually start them all. Data validation: We could've just blindly started all the databases and trusted they worked, but we didn't trust them. We ran innodb tablespace checksum checks on everything, and also did backups of a lot of the databases before we even tried to bring them back up. (the act of bringing them back up modifies the tablespace, and we didn't want them messing themselves up worse, so a pre-backup was for paranoia...) MyISAM vs InnoDB: When you lose power to a MySQL db w/ MyISAM tables, the indexes are generally messed and you need to rebuild. Fortunately almost all our databases are purely InnoDB nowadays, so that wasn't a huge problem. Unfortunately, though, the global DB (which is required even to get the site up in partial mode where some users are available and others aren't) is still like 5% myisam and we just hadn't got around to converting those few remaining tables to innodb yet. So every machine in the global cluster required index rebuilds and data checks. That was annoying. The Chef user cluster was also MyISAM (our last MyISAM user cluster), so rather than trust Chef, we restored from an old Chef backup and replayed binlogs to catch it up. That took some time. Disk cache issues: We have battery-backed RAID cards with write-back caches. That means the RAID card immediately acknowledges writes and tells the OS (and thus DB) that they're done immediately, before they're on disk. This speeds up the DB. But if you lose power, those writes would normally be lost, which is why we have battery-backups on all the cards, and we even monitor the battery health w/ our automated checks. But unknown to us, the raid cards didn't disable the write caching on the drives themselves.... which is frickin' useless! If the controller is already lying to the OS (but doing it safely!) why should the disks behind the controller also lie, but unsafely, for minimal benefit? Our bad there. We should've had that right. So a couple machines were just gibberish afterwards and had to be restored from backup and had their binlogs replayed to catch the backups up to present. Binlog syncing: We weren't using the option to sync binlogs to disk, so we lost a small number of transactions right before the power loss in the case of clusters that we had to restore from backup. Regrettably, we won't be able to get those posts or comments back. Slaves tuned for speed: A lot of slave servers (mostly in the global cluster, since the user clusters are almost all master-master now) were tuned for speed, with unsafe filesystem/sync options that favored speed over reliability. Which is normally okay, since you'd lose one machine, not all of them. But we lost all of them, so restoring them all from good slaves was time-consuming. Things we're doing to avoid this crap in the future... We're: -- getting working NICs in those 9 machines -- all our DBs have redundant power supplies. we'll be plugging one side into Internap's, and the other side into our own UPS, which itself is plugged into Internap's other power grid. that way if EPO is pressed, we'll have 1-4 minutes to do a clean shutdown. (but if we do the rest of the stuff right, this step isn't even required, including having UPSes... in theory... but the UPSes would be comforting) -- disable disk caching behind all our RAIDs. (bleh... wanna kick ourselves for this, but also the raid vendors for even defaulting and/or allowing it in a BBU write-back setup) but also testing all existing and new hardware to make sure data makes it to disk and pulling power in the middle of write-heavy operations, then verifying the resulting disk image later with the expected result. -- finish our MyISAM to InnoDB migration, so we don't have to deal with MyISAM index rebuilds -- enabling binlog sync options -- stop tuning slaves for speed. this used to matter, but we don't really do the slave thing as much as we used to, so the gain isn't worth it. -- user level backup harness. we already have a tool to backup a single user's to a GDBM file, incrementally. (so if we run it a day later on the same file, it only updates the changes) so we plan to wrap a major harness around that tool and be backup up all users, all the time. this means that in the event of a future major corruption/outage, we'll be able to restore user-at-a-time, and even to a different database cluster than the one they came from. this also means we can prioritize recovery based on account status, popularity, attempted activity, etc. (and yes, we'll continue doing system-level backups as well, but it's good to have data in different formats just to be paranoid...) -- also, we already bought a bunch more disk space that we installed today, so we have more room to do backups for bizarre non-typical reasons, and don't need to compress/shuffle so much stuff to make room sometimes. |
| Tuesday, January 18th, 2005 | |
| 5:04 pm [j4k0b] |
How does the S2 code cleaner work? I assume the content is piped through some program and stripped of malicious code, but how do you know which code is malicious? And where is that program located? I searched the CVS repos and the manual but couldn't find it. I'm interested in doing something similar with PHP. |
| 11:53 pm [evan] |
link relationships Brad's already pledged support for Google's new initiative for preventing comment spam, but I thought I'd bring it to all of your attention. Also, it occurs to me that the attribute might be useful to prevent userinfo spamming: if you put rel='nofollow' on all of my friendofs, then my userinfo page isn't giving them credit. (If they are my friends and deserve credit, I'll link to them as friends anyway.) (I'm not speaking for my employer, just a friendly thought.) |
| Thursday, January 13th, 2005 | |
| 5:29 pm [jproulx] |
Anonymous comment spam There's a new patch in CVS that disables all links in comments from anonymous posters: URLs from links will be displayed adjacent to the link text, and bare URLs won't automatically transform into links. The motivation behind this is to further curb efforts from spammers that are just trying to up their clickthrough ad traffic and search engine rankings. This is only the first part -- we'll soon add a new talkprop that "acknowledges" an anonymous comment, triggered either by unscreening the comment (if it was screened) or replying to it. Acknowledged anonymous comments will then show the links as intended. It's not live yet because we want to gauge your initial reactions -- if we need to we can add a per-user option to turn this off, but we hope that's not necessary. |
| 4:38 pm [mart] |
XMPP PubSub on LiveJournal Today I've been reading about XMPP PubSub and now have a general idea of how it works in my head. I'm still not entirely convinced that it's the best way to shift large volumes of entry data between the big sites, but I do think it'd be worthwhile to implement it anyway. The nice thing about XMPP PubSub is that we can implement it in two parts: The PubSub consumer seems like a good place to start, assuming there are actually data sources we can subscribe to for testing. It looks like the PubSub.com feeds aren't suitable because the client must connect directly to their server to do it, and they only serve up their own feeds. If I'm understanding correctly, the pubsub consumer daemon will need to have an account on a Jabber server through which it sends and recieves messages. LiveJournal.com has the resources to run its own Jabber server quite easily, and others who don't want to run their own could presumably just use an account on jabber.org or whatever, so this doesn't exclude anyone. If anyone wants to add anything or correct me, please go right ahead… |
| 11:40 am [banana] |
E-mail forwarding I have a support request open for a problem with e-mail forwarding. I'm not sure if/when/how it would come to the attention of delevopers, so this is a bug report. It's still happening. I can provide headers from a mis-directed message if anyone wants to see them. |
| Wednesday, January 12th, 2005 | |
| 11:41 pm [drumguy2002] |
|
| 4:09 pm [laochbran] |
Beyond a server-based content distribution network ![[info]](http://web.archive.org/web/20050207063625im_/http://stat.livejournal.com/img/userinfo.gif){kind=small} mart has raised some very interesting ideas about how to integrate the social networks of journaling sites. The models he has proposed are sensible, logical extensions of the way Usenet was built, however they introduce new challenges in terms of server performance. To be specific, a bunch of traffic between servers is created and managed so that content is duplicated around the place. This creates a number of challenges:Sites lose control over who receives served content. If my friends-only entries get copied into a cache at DeadJournal, then I am now trusting DeadJournal to maintain my security as well as trusting LiveJournal. Ideally, the number of trust relationships anyone has to enter will be minimised. Sites serve content from other sites. Proper payment for hosting can become problematic if someone establishes a primary relationship with one site, but most of their content gets served by another site. The most likely consequence is that big sites like LJ will be paying for even more bandwidth and disk, subsidising smaller sites. This then requires the establishment of chargeback mechanisms or other cost-recovery devices, and the difficult politics imposed by the trust issue get even worse. A fairer system would be one in which sites serve their own content and nobody else's, and yet are able to provide a unified friends page. This can be achieved by separating the aggregation function from the content-delivery function, and using client-side includes. The system might work like this: I send a request to LiveJournal for my friends page. LiveJournal sees that my friends page is 50 entries long, and finds the ID and timestamp of 50 entries that it might serve to me. It then sends requests to TypePad, DeadJournal and GreatestJournal containing the lists of friends that I have at those places, and a timestamp range. TypePad, DeadJournal and GreatestJournal send URLs (but no content) back to LiveJournal for any entries that meet those search criteria. LiveJournal then generates a friends page that contains client-side includes for the entries that are assembled into my friends page, and my web browser is then responsible for fetching the entries back from the various sites where they live. (Would this method increase the ability of my web browser to cache LiveJournal entries locally and reduce the load on LJ when I refresh my friends page? That's a fun possibility.) In version 1 of the protocol, it might be best to get only public entries from aggregation partners. Version 1.1 could involve incorporate an assertion of my identity to those sites, allowing more secure entries to be distributed to me. The weakness of this method is the delay introduced by the dynamic query to other sites to fetch a list of content, the possibility of timeouts and so forth. Ideally, this could be managed with a highly-intelligent client. Alternatively, it simply becomes a marketing opportunity for our Evil Robot Overlords (who have been nice to support people already - I like that a LOT) who get bragging rights over the fast integration between TypePad and LiveJournal, but can say "Integrating with sites outside our Evil Corporate Alliance may delay processing of your friends page." If the version 1 (non-secure) version of the protocol is implemented, it should be possible to cache data pretty effectively. Perhaps LJ and TypePad can market an ultra-clever browsing client that does a lot of this integration at the client end to match their ultra-clever posting software? Current Mood: creative |
| Tuesday, January 11th, 2005 | |
| 6:49 pm [mart] |
Content Distribution Network While thinking about the problem I talked about in my previous entry, it occured to me that it is quite wasteful for every site to have to talk to every other site. Instead, we can borrow from the USENET model and create a structured distribution network. For example:  A completely hypothetical network layout, of course. The basic principle here is that each node has a set of peers and keeps track of which of those peers are interested in each journal. Subscription control messages as well as entry state changes are passed around the network through these channels, and since the links are created through co-operation between two nodes they can either be persistant sockets or pull-type connections depending on the needs of the two peers. Nodes must also track which journals should be forwarded on to neighbours, to avoid redundant forwarding and ensure that smaller sites don't get overwhelmed with data. All of the nodes need to know about all nodes which produce content. To avoid nodes tampering with the data as it passes through the data is signed and each content-producing site has its own keypair. Key exchange is the tricky part, as it is the only part of the process where every node must connect to every other node directly so that everyone has everyone else's keys. As you can imagine, this is a closed network as it requires co-operation between nodes. This is much like USENET, but the network will be a lot smaller. The obvious question is "What's in it for the sites?", and that is a good question. Big sites benefit from reciprocal links because they are trading valuable content, but bigger players have no real reason to let the little players in. As distasteful as it may seem, someone has to pay for these things, and so the worst case is that the USENET model is followed where a peer pays an upstream provider to let it feed from them. This isn't really that bad, as a bunch of smaller services can co-operate together to get a single link to the main network and share costs between them. I think this, really, is the only feasible model for now. If we design it right it could be general enough to let producers and consumers that aren't LiveJournal-based in later, such as (for example) TypePad pushing content into the network via LiveJournal, and aggregator peers which suck data from a bunch of RSS feeds and republish them onto the network as well as user-oriented aggregators which only consume content and provide something not unlike a LiveJournal friends page for those who don't have any wish to publish but want to read. That's for the future, though... for now, it'll probably just start as a small network between LJ and DJ and perhaps Plogs.net. What do you think? |
| 2:50 pm [mart] |
Efficient Linking of Weblog/Journal Sites It would be very cool if certain aspects of the social networks created by the different LJ-based sites could be absorbed together. Right now we have the rather lame solution of adding RSS feeds from other sites, which taken to the logical extreme means that all of the sites end up sharing one namespace, but all of the journals have different names depending on where you are. This is a mess. RSS works okay for pulling in content from outside, but when it comes to other LJ-based sites we could do so much better, and do it much more efficiently. I should be able to add scsi@deadjournal.com to my friends (or "watch"!) list on LiveJournal and have the entries from that journal appear on my friends page. We talked about this a few years back, but the situation has got a lot better since then as other people have done some of the things we would have had to do. On the surface this doesn't seem too hard: we need some efficient mass-transfer protocol so that the sites can pull updates (create, edit, delete) in batches between each other, and some way to express what journals each site wants. Sending the list of journals over each time would be a bit lame, so each site could instead maintain subscription lists for the other sites it can link with. Of course, this requires co-operation between the different sites, so it's not brilliant. Also, I'm not entirely sure if we have the right data to be able to distinguish the create, edit and delete operations... but syncitems does this for a single journal, right? Since these special accounts won't have any of their own journal views, the entries can be safely deleted once they're too old to appear on a friends view. By now the Atom folks might well have something we could use for this. Back when we originally discussed it I was pushing for creating our own XML format, but that was before the Atom folks came along and did basically what I was proposing. However, last I was tracking Atom they were still trying to decide on the XML format and not really near any kind of API for pushing content around in an organised fashion. Obvious caveats: need to be able to comment on other sites with a LiveJournal account and vice-versa (decentralised TypeKey-style auth fixes this), handling security-locked entries securely (can't without an user-trusts-site relationship), knowing how many comments there are on the remote entry (or just make it say "Read Comments" for those)... and do we let (say) DeadJournal users post in LiveJournal communities? Should communities be shared too? Yeah, this entry doesn't have any concrete stuff in it. I think it's worth working towards a proper design for this, though, as it's something we've talked about forever. |
| Saturday, January 8th, 2005 | |
| 7:45 pm [bsdguru] |
MogileFS and Perlbal under FreeBSD Seeing that I have issues with the mailing lists I'm going to post this here. I'm busy looking at running MogileFS and Perlbal on FreeBSD servers but there is specific code which relates to AIO::Linux and various other bits and pieces which is included with Perlbal which currently does not allow one to run these systems under FreeBSD. Are there any plans for Perlbal and MogileFS to run under FreeBSD or use non portable perl modules in future? I'm busy working on a pseudo web proxy for the FreeBSD ports system to use which would be located on slow leased-line sites which have multiple boxes to save b/w by reducing the number of times distfiles are downloaded when they've already been fetched once. Current Mood:  tired |
| Friday, January 7th, 2005 | |
| 7:48 pm [bradfitz] |
Six Apart and ownership of code I know some of you are still apprehensive about the Six Apart acquistion, so I wanted to give you guys some updates. Six Apart doesn't want to own your code. They don't even care too much about owning Danga's code. They just want a right to use it, and they want to make sure somebody is able to legally go after any company that tries to violate the GPL'ed code by selling server software based off LiveJournal code without releasing their modifications. (no, this does not mean attacking DeadJournal, because DeadJournal doesn't sell/distribute their changes, and DeadJournal even gives stuff back from time to time....) We're in talks now with the Free Software Foundation to figure out how to best do this. (well, they sent us some mail offering help and we replied saying that'd rock, but that was just today, so "in talks" might be overstating it... but we're working to start talks) Under our old pre-SixApart TOS it said something like we own all your contributions. That was just boilerplate stuff that I personally never wanted it in there, but I wasn't good at legal documents, so I left it. When SixApart bought the company they left that in, but their lawyers changed some words a little to make it clearer. So while technically SixApart could arguably own your contributions now (if you agreed to the TOS), they don't want them. We want you guys to own them, as long as we have a license to use them. (under the GPL, Artistic, BSD, whatever....) I'll give you guys an update when we're further along in these talks. Probably next week ... not this weekend. Future LiveJournal contributions People think we're ditching this codebase and moving to TypePad or Movable Type. You know how hard that'd be? Think how many LiveJournal features and formats we'd have to port. We love the LiveJournal code and associated codebases and you're still going to see us hacking on them, adding new stuff, fixing stuff, etc. To come.. More announcements when we know more. Feel free to flood us with questions and we'll answer. We don't want anybody left in the dark about stuff. |
| 4:03 pm [bradfitz] |
partial ack tcp corruption bug Thanks to all your bug reports and especially the tcpdumps, F5's been able to find and fix (in only 3 hours!) the TCP corruption issues you've all seen. The new code is running now and should fix the problems you've been seeing. Please report any problems that happen past this point. And with pcap files, ideally... thanks! (FYI: we're running some pre-release BIG-IP code because we do some bizarre HTTP and load balancing stuff and they wanted us to test it...) But it's totally worth it. We can do things like: ( TCL config code... ) |
| 5:05 pm [mart] |
Corrupt Data Something is corrupting data between the web servers and my client. The most obvious symptom of this is that userpics and picpix/pics.lj images are getting distorted, but it's also corrupting the gzip-encoded data streams on journal views and causing pages which have all of the right characters but in the wrong order. It's intermittent, though; refreshing usually fixes it or at least changes the nature of the corruption. Several other people have mentioned this, too. Given the timing, I'm guessing the blame lies with the new network configuration. |
| Thursday, January 6th, 2005 | |
| 10:55 pm [telyio] |
Typekey. With the merging of Six Apart and Livejournal, will we be seeing Typekey integration with Livejournal comments? This would be the sort of thing that I've been waiting to see from Livejournal for years, and hope that this is being considered. |
| Friday, January 7th, 2005 | |
| 2:12 am [vasaki] |
multi-dimensional categorization I know that this topic has already been discussed several times here, but nevertheless. I don't use LJ or any other popular blogger for publishing. Let me explain why. I personally could use blog for one reason only - I view blog as knowledge base with possibility to make certain parts of it accessible to people. The concept of "Knowledge base" is very much disputable, I see it as a carefully classified information snippets collection. The key thing here is "carefully classified". A simple categorization is not enough, each snippet has to be classified in several dimensions. For instance, I might have certain pieces related to IBM, and other pieces related to HPC (high-performance computing). Then, the article about "Deep Blue" I would assign to both classes - IBM and HPC to make the future references easier. It is a very modern area of research nowadays - ontologies, Semantic web, semantic blogging and the like. However I don't think blogging needs all that technology now, all it needs is a multi-dimensional classifiaction, as I described it here. A certain central structure of available classes is also nice-to-have. For instance, in addition LJ could contain a central list of keywords, meaning of which are unambigous, which people can use as classes for their postings. For example, central directory could contain a tree-like list like this - Companies - IT - IBM - Sun Microsystems - Microsoft - SAP - FMCG - Nestle - Kraft - P&G; - IT area - ERP - Web Thus, I could list, for instance, everything classified as "Microsoft" and "ERP" to find out any activity of Microsoft on ERP market. This is more close to so-called "Semantic blogging" and I tend to think it to be quite utopian idea, and not urgent at all. Simple multi-dimensional user-specific classification would be enough for the beginning. Of course, to make any use of such classifications, we need to add some sort of query language. Links: http://nudnik.ru - russian blog engine that is exactly the concept I'm talking about (check small keywords after # sign. Some posts contain several keywords. http://www.livejournal.com/community/lj http://www.livejournal.com/community/lj http://jena.hpl.hp.com:3030/blojsom-h |
| Wednesday, January 5th, 2005 | |
| 11:48 pm [fg] |
more HTTP Here is another observation on how HTTP replies are getting mungled: If I disable Keep-Alive in the connection request, and only supply one request per connection, things seem to go a lot smoother. Chaining requests in one connection seems to exacerbate the issue. I don't know if the fix is complete, but it seems to be working for now. Also, someone pointed me to a post about blocking ljArchive. Is there something I can do to make the client more friendly to the servers? |
| Sunday, December 26th, 2004 | |
| 4:01 am [ibjhb] |
Nesting Comments What is the determination for not showing full comments? If you go look at posts that have a large number of comments, some of the comments only show the subject, who posted and when. How does the LJ server determine if it should show the full thing or not? Current Mood: productive Current Music: Story of The Year - Anthem of our Dying Day (Energyradio.FM - Energy X) |
| Tuesday, December 21st, 2004 | |
| 10:28 pm [cemcom] |
Membership lists for communities with more than 500 members I've been doing research on visualizing the LiveJournal community space using implicit links between communities. In particular, I'm using the number of members two communities have in common as a weighted edge between those communities. This type of visualization seems to be useful in making salient the strength of inter-community ties and visualizing possible relationships between communities (e.g. mediatory links between two communities-of-communities). A brief write-up of the research (with pictures!) is here: http://cemcom.livejournal.com/7511.h A large issue with what I'm doing, however, is that I'm not able to get membership lists for communities with more than 500 members. These communities doubtless play key positions in the LJ community-space, and any analysis failing to include them is probably incomplete. The community info page (which is what I've been scraping, because it has more of the data I wanted to collect, compared to the FOAF feed) is limited to displaying 500 members. Is there a way I could get this data? I would be willing to write any code necessary, if that is helpful, though it seems like a simple SQL query would take care of my needs. Thanks much, Eugene |
| Sunday, December 19th, 2004 | |
| 5:25 pm [fg] |
HTTP/xmlrpc Hi all, I'm seeing strange things happen when I call GetEvents through xmlrpc. Sometimes the connection simply closes with no response Other times, I get what seems like otherwise proper XML, but with strange strings inserted like this: HTTP/1.0 200 OK Date: Mon, 20 Dec 2004 00:12:54 GMT Server: Apache Set-Cookie: ljuniq=5HTnztLtYkRSwZ1:1103501574; expires=Friday, 18-Feb-2005 00:12:54 GMT; domain=.livejournal.com; path=/ Content-Length: 228115 Content-Type: text/xml SOAPServer: SOAP::Lite/Perl/0.55 Keep-Alive: timeout=30, max=100 Connection: keep-alive image/png <?xml version="1.0" encoding="UTF-8"?><methodResponse><params>...what is that 'image/png' doing there? I put the request that generated the above reply here in the ( cut ) Any suggestions? Is there something suspect with my request or the request's http header? |
| Friday, December 17th, 2004 | |
| 8:55 pm [commie_coder] |
hello from a php hacker hi all apparently when you join you're supposed to introduce yourself, so here goes. i've been working in php/mysql for the past couple of years, and i've decided to learn perl and particularly mod_perl. so, i'll try and contribute to lj development as part of my learning process - hopefully it will be a win win situation :-) i'm quite new to perl, but not new to programming or web development, so hopefully i'll be able to take something on sometime soon. thanks justin |
| Sunday, December 12th, 2004 | |
| 7:04 pm [triptrashbang] |
Meet the Demand for Categorization Hello everyone, This is my first post to the ![[info]](http://web.archive.org/web/20050207063625im_/http://stat.livejournal.com/img/community.gif){kind=small} lj_dev community. I have been using LJ for about two years and support LJ through paid membership.I used to use other blogware such as b2 and Movable Type; I joined LJ because of the user-base makes it easier to connect to other people. Like most other experienced LJ users, however, I find myself running into problems that have yet to be addressed by the LJ feature set and are being circumnavigated with methods that aren't necessarily the "healthiest" ways to propagate communication within an online community. I've seen a lot of people start wondering why these features aren't implemented in LJ. In particular, I'm talking about the most common problem of categorization of entries and subscription access to those entries. The demand for this feature within LJ has been made obvious not by the direct demand of such features but by how people are adopting methods of dealing with the lack of such features within LJ. There are two ways of dealing with lack of categorization in LiveJournal: The "Friends Only" Journal and the "Multiple Account" scenario. Both methods have serious drawbacks and are not the best way to maintain healthy connections online. The "Friends Only" approach is exclusionary, and the "Multiple Account" approach creates extra management overhead on the part of the user and that user's friends. (I would get into more detail about the problems of both methods of approach but I believe that would be beyond the scope of this community.) Anyone not using either of these two methods may opt to create group filters for specific entries, but then runs into the problem of playing "Blog God," having to decide and keep track of who really has access to each post. This is okay when we get to know people through interaction, but becomes a monumental headache when we've just added people to our friends list or are unsure what may or may not be appropriate for all readers. The other option that most people adopt is the "all or nothing" approach, where everyone on the friends list gets to read every entry posted by the user. The problem with this methodology is we are undoubtedly creating bad surfing habits--people left without choice of what to read skim through it all and until something "interesting" hits them. This psychologically objectifies all users into faceless, soulless beings, as what we experience is turned into commodity for mass consumption. Often this leaves both ends of the experience feeling empty, often turning to the aforementioned "friends only" blogs after feeling used and exploited. An obvious solution to meet the demand of what one might call selective reading (similar to a pick and choose market?), is to allow categorization of entries and allow user-subscription to such entries. Created properly, this feature has the ability to become very powerful for the following reasons: 1. Users can maintain one account and not worry about having diverse interests, 2. Each category can be subscribed to, thus eliminating the need for the user to be exclusive or ignore other users (unless one really must)--in other words, it nurtures the original premise of the internet as it turns it back towards a more "sharing" atmosphere, 3. Eliminates the need for a "Friends Only" journal, as users can feel comfortable posting to categories that are not sensitive, allowing access to a larger "friend" base, 4. Allows users to be selective of both who's journal they are reading and what they are reading about, gives the power to the user to select from what another user is already sharing, 5. Depreciates the "fan lists" by allowing users to gradually get to know other users as they interact, rather than subscribing to an "all or nothing" instant-gratification method of adding friends 6. Can be used as a "front-door" mechanism on the user-info page, and each category can have it's own interests--thus making it easier to choose which categories/interests to share with others. There are a lot of very dynamic possibilities with a categorization system within LiveJournal. I haven't even really scratched the surface. But I think that if LiveJournal doesn't get on the ball with this soon, someone else will rise to meet the demand of this type of journaling methodology. I believe the time has come to incorporate human psychology into the software, rather than letting the software dictate how we communicate with others. (I believe it's creating a lot of dysfunction, perhaps?) Regards, johnny :) Current Music: P h i l o s o m a t i k a |
| Wednesday, December 8th, 2004 | |
| 9:03 pm [andrewducker] |
Friend Count I was reading recently about The Long Tail and the clustering affect that occurs where you get an inverse relationship between the number of people that have X friends and X. (5000 people have 1 friend, 2000 people have 2 friends, etc, etc. until only a few people have 200 friends). Anyway - that data isn't easily accessible, so I was wondering if it was possible to put together a graph where X was "number of friends" and Y was "number of people having that many friends". If anyone with access has 10 minutes free and a graph-making program handy. that is :-> |
| Monday, November 29th, 2004 | |
| 6:28 pm [perlmonger] |
internal/external links (and hello) Hi, I'm Pete Jordan; been reading here for a while, now joined because I want a change that, as far as I can see, I can't kludge though S2. I'm in the process of reskinning my LJ, as far as possible via a stylesheet on my own server but via magazine-based theme layer hacks where needed. One thing I want is to visually distinguish internal (LJ) and external links; mostly this is ok, but I've not found any way to so distinguish poll links. So. What are the chances of mangling the poll rendering code to either wrap the whole thing up in a div with a (say) "ljpoll" class or (better from my POV) generate all internal links with a class of "lj"? This latter is what I'm doing in my theme layer code. I'll gladly contribute a perl patch, subject to my lack of tuits. |
| Sunday, November 28th, 2004 | |
| 10:53 pm [mart] |
“Killfiles” Users clearly want the abilty to ignore every entry and comment from a given user, even in communities. It seems that people are now resorting to doing this in S2 layouts, which is clearly the wrong place to address this. It seems to me that implementing this kind of thing shouldn't be too hard. The main problem is that it's hard to filter out items without reducing the number of items present due to the way the data fetching works, unless the filtering goes in right in the guts of the API, which I think is the wrong place to put it. I'm a bit rusty on this stuff since I've not been doing much other than S2 for a while now, but I'm thinking perhaps a couple of new API functions which can be called at some point in the data-fetching process to filter out entries and comments respectively. Depending on where in the proceedings this is done, I expect that the input to the former would be a list of entries and the latter would be a tree of comments. There's a Zilla bug filed about this already, but if there's any discussion to be had then let's have it here since Bugzilla sucks for discussion. |
| Wednesday, November 24th, 2004 | |
| 7:20 pm [mart] |
Little Toy Today, as an experiment, I wrote a little tool to translate Xanga's “Skins” to LiveJournal S2. The results aren't spectacular because the source data isn't, but one day perhaps I'll use this code to make an LJ-specific template system as simple as Xanga's. You have to join Xanga to access the skins directory, but bugmenot can probably help out there. |
| Friday, November 19th, 2004 | |
| 2:47 am [omni1100] |
More Journals Per Person Hello Folks, I am very new to this whole deal, and have just started looking in to it. I am posting this question here, but I am not entirely sure that it is in the right place. I'm sorry if this is incorrect. Basically, I am looking for an "addon" to the LiveJournal code. Is there some sort of website that lists "mods" or addons for LiveJournal? Or do mods not even exist in this community? What I am looking to change is very simple. I think the core code in LiveJournal is great. But what I would like to be able to do, is make it so that each user can have more than one journal to their username. The way it is now, if I want to have more than one journal, then I need more than one username. I need one username per journal. But is there a way to make it so that each user can put as many journals under their username as they'd like? Again, I'm sorry if this is in the wrong place. Any replies would be very helpful. My email address is vincec@rochester.rr.com. Thanks very much, - Vince |
| Friday, November 12th, 2004 | |
| 12:09 am [mart] |
Poor old Zilla It's broken again. |
| Tuesday, November 9th, 2004 | |
| 5:26 pm [lightmanx5] |
More on RSS I noticed that I could input LiveJournal specific tags (as long as they were of "<lj user=" type and not "<lj user=" type). Also...I made some changes to my XML file... and I noticed that LiveJournal will reflect some of these changes (after it updates itself to the file of course), but sometimes it won't. I was wondering if anyone knew the process therein, and could describe how all this stuff works. Thanks so much for your time! Current Mood:  amusedCurrent Music: Fall River - At Least You Sent Flowers |
| Monday, November 1st, 2004 | |
| 12:22 pm [mannu] |
Why does LiveJournal strip off HTML comments? I'm trying to implement category-specific RSS feeds for my journal, by reading in the main feed, and filtering based on metadata embedded into the posts -- in the form of HTML comments. Unfortunately, LiveJournal strips off HTML comments. :( Why? I've written about it here on my journal. Any suggestions on how to implement this? |
| Tuesday, November 2nd, 2004 | |
| 5:36 pm [fg] |
export comments Hi folks, Did export_comments.bml break? I'm getting empty replies when I request the url. |
| 11:32 pm [legolas] |
3 posting dialogs UI design error. I think the quick comment's UI and the preview page's UI is wrong. Both look very similar (2 buttons and a checkbox), these things present the same basic options (post, preview, and spell check), but they are very different. To make matters worse, the quick comments preview leads to the normal comment page. In fact, the buttons at the bottom of the 'new entry' page present yet another way to again present the same three functions (this time with three buttons). On the quick comment box: [post comment] [more options] [checkbox meaning: don't post comment when you click post comment (!!!) but preview and check spelling instead] On the preview page: [post comment] [preview] [checkbox meaning: also check spelling when you click preview] On the New Entry page: [Update Journal] [Spell check which also previews!] [Preview] You know the jokes with the button you have to click but that keeps jumping to a different location when you approach it? Well, this isn't all that different, except this is not meant as a joke! |
| Sunday, October 31st, 2004 | |
| 10:59 am [mart] |
Phone-posting to Other People's Journals Last night I made a phone-post but I think I mistyped my caller-id and/or PIN and managed to hit someone else's journal instead. I don't know where it went, but it said it was posted successfully and yet it is not in my journal today. I think it would be useful if the phone gateway quickly spelled out the name of the journal associated with the callerid and PIN you entered so that you can see if you got the right one. Of course, this does have a major security implication: if you do mistype and manage to hit someone's journal, you then know whose journal you've got access to which might cause malicious people to cancel their current phonepost but make a note of what they typed in for later abusing. While on the subject, I'd also like to optionally have the phone gateway tell me what it thinks my callerid is when I press the “pound” key to auto-login, since despite trying several combinations I still haven't found what LJ is seeing as my callerid when I call from a British number through my unofficial 0870 (national rate) access number. If I didn't have to type the callerid in every time there'd be less chance that I'd get it wrong in the first place. |
| Saturday, October 30th, 2004 | |
| 10:26 pm [legolas] |
I have something weird (mozilla 1.7.3), that I think is new: Open a page with comments to an entry. Use the top 'post a comment' link. With the quick comments feature on, a box appears in the page. Enter some text, check the 'Check spelling and preview', then click the 'post comment' button. On the preview page, use the back button in the browser, then click the 'post comment' button again. You'll end up at an error page. (Workaround: click on the 'post a comment' link again, and your comment box will resize a bit, and then you can click post comment again.) Can anyone confirm that this isn't just me? (Discovered while posting a comment on an lj_backend post while lj was acting up.) |
| Thursday, October 28th, 2004 | |
| 3:27 pm [lightmanx5] |
Syndication Is there a section of the LiveJournal website that deals with developing your RSS/ATOM feed specifically for LiveJournal? I'd like to learn how to generate my RSS/XML document so that it does what LiveJournal is looking for, you know what I mean? (If you want the current username of the syndicated account, go ahead and ask, but I don't mean to "promote" it here.) For example, there's obviously all this (link) in the FAQ about Syndication, but it doesn't tell me the developer-type information I need to know. I want to know things like, what does LiveJournal do with this piece of code: <image> Current Music: Falling Cycle - I Still Dream (Part 1) |
| Wednesday, October 27th, 2004 | |
| 12:15 pm [benfranske] |
Problem with vhosts [Error: Irreparable invalid markup ('<virtualhost *:80>') in entry. Owner must fix manually. Raw contents below.] Please excuse me for posting here, but I've had this problem in lj_everywhere for a few weeks now and haven't been able to get it resolved, so I'm hoping someone here can tell me what I'm doing wrong. I'm trying to run LJ as a vhosted subdomain on one of my servers, but it takes over all the vhosts for some reason. Here's a snip from my httpd.conf: <VirtualHost *:80> ServerName blog.edinahigh.org PerlSetEnv LJHOME /home/lj PerlRequire /home/lj/cgi-bin/modperl.pl </VirtualHost> <VirtualHost *:80> ServerName mail.franske.com DocumentRoot /home/www </VirtualHost> For some reason if you go to mail.franske.com you end up with the LJ page even though you should be getting /home/www. If I comment out the PerlRequire line in blog.edinahigh.org everything returns to normal and vhosts work as expected (except of course you can't get to LJ). The modperl.pl file seems to be overriding my apache config. Any ideas? |
| Tuesday, October 26th, 2004 | |
| 11:16 pm [crschmidt] |
RDF Data on LJ As an RDF nut, I love seeing more data exposed as RDF. LiveJournal is one of the world's largest sources of RDF data, in the FOAF it exports, and in many ways tends to be on the leading edge of new ways of exporting data for use by its users. However, in a few ways, LJ is lacking, especially in the RDF department. These are all relatively simple changes, which I would be totally willing to submit patches for, but I'm not sure that there's any want for more RDF on LiveJournal. I know that I want it, and I know I've been asked by several people why LJ doesn't export RDF-based RSS feeds. I'd like feedback. Should I go ahead and work on patches to increase RDF output? These things would basically correct current data or lie alongside current options, and would only be used by tools which understand it, most likely. (Of course, there's also the fact that users might see others and get confused, but really, users almost always end up confused in some way or another.) Can I get either an "Amen" or "Shut up and go away" for more RDF on LJ? |
| Thursday, October 21st, 2004 | |
| 8:27 pm [crazyscot] |
So, about this PhonePost issue...? Q. When is a WAV not a WAV? A. When it doesn't have a RIFF WAVE header. (Since I drafted this post I've had a search around and saw that there is a "known issue with phone posts" in the Support yellow box, plus a few open support requests saying phoneposts don't play, but I can't seem to find discussion of what that issue is. [I have no support privs.]) To cut a long story short, a friend tried out PhonePost today, and I wanted to hear what he had to say. The alleged WAV file he recorded carried a MIME type of audio/wav when I downloaded it, but clearly wasn't one; poking into it, it didn't even have a standard RIFF header (which specifies the sample rate, encoding type etc) - it just seemed to be garbage. (There's a possible related issue here, in that I think my friend specified Ogg Vorbis format for his phone posts, and that was ignored; but he's best placed to take that up.) I tried a few plausible options to sox but didn't get anywhere.I poked around the LJ CVSweb for clues; eventually downloaded the libgsm-tools Debian package and was able to use tcat to convert the file into a raw μ-Law which I was then able to pass to sox to play. (tcat -u -p foo.gsm | sox -r 8000 -b -U -t raw - foo.wav was the rune I figured out.)At the very least, it would be nice if the LJ backend would add proper RIFF headers to these files. (I'm assuming GSM is indeed a valid compression scheme for the WAVE format; if it's not, then please stop misrepresenting them!) If GSM is intended to be the "way forward" for non-Vorbis phone posts, then judging by the support board and my experience today there seems to be an urgent support issue to resolve here, or at the very least an FAQ to write on how people should play the darn things. |
| 7:17 am [mart] |
Zilla is broken Broken is Zilla. |
| Sunday, October 17th, 2004 | |
| 8:04 pm [thefowle] |
mypost.html?ljuncut hello all. been idling here forever, and i have something useful for once i actually tried doing. i have a really badass thread on semantic web & niche media markets but i wanted to post a link to the page which would show the far-more-concise lj-cut version. i resorted to posting to the day of entry. the basic premise is i can do mypost.html?ljuncut and it'll look like it would from a friends page or from a journal page. this is a simple patch to take the arguments and feed them into LJ::CleanHTML::clean_event from within EntryPage_entry. this is completely untested code in every manner possible, but i believe how i think would be done. its near the tail end of a long call stack - very nice form their lj coders, thanks - and fairly autonomous, so it looks ok. on line 262 of the august 25 2003 snapshop, insert: # show a cut edition of your page in post form. if ($opts->{'getargs'}->{'ljuncut'}) { $entry->{'props'}->{'opts_preformatted'} } ideally it should also display something indicating that it is a cut page? first time really digging through lj code. very nice. took a bit of cross referencing (mainly verifying that getargs was in fact the uri's arguments, but far less effort than expected. well done! |
| Tuesday, October 5th, 2004 | |
| 12:12 am [ashley_y] |
Trusting Styles Is any work being done towards the concept of trusting other people's S2 styles? Consider: you've friended me and I happen to look at your friends page. On it I see my own entries, including private ones, but in your style. Unbeknownst to me your style "reports back" all my private entries to some server by loading images with my private data embedded in the URLs. Now, I trust the official styles not to do this, of course. And I trust my own styles. But I don't trust other styles... |
| Saturday, October 2nd, 2004 | |
| 5:52 pm [cemcom] |
Community Membership Data Hey all, I'm working on doing some research with LJ community membership data. For communities with 1000+ members, FOAF data is incomplete and the Community Info page has a link to the directory, which I know is generally off-limits for scraping (not that I'd want to write the code to scape it anyway). Anyone have any suggestions for how I can get the full list of community members? Also, there doesn't seem to be an easy way to get 'member of' data w/o scaping User/Community Info pages... That data's not in FOAF or in the Friend-Data (fdata.bml) outputs. Do I just scrape the info page, or am I overlooking something? Thanks much. |
| 1:55 pm [whitaker] |
Community Rename Security With the resolution of Zilla Bug #593 and the subsequent disabling of community logins, communities are no longer able to use rename tokens, since that page requires the target user to be logged in. A user complained about this one day because they were able to purchase a rename token for their community, but then had no way to use it. 2 solutions: 1) Disable purchasing of rename tokens for communities (lame) 2) Add authas support to /htdocs/rename/use.bml (good) The problem comes when we decide what the policy should be for using #2. Who should be allowed? Any maintainer? Only the original user? Some other criteria? We've been discussing this internally for a few days now and the basic consensus is that any community maintainer should be able to do the rename... and the responsibility will be on communities to only have maintainers that should actually have full admin access to the community. Obviously the decision made here will eventually have repurcussions throughout the rest of the site, as we run into this problem in other places... so I wanted to make sure this was well thought-out. All of the staff members seem to agree that allowing any maintainer is the right thing to do, but I just wanted to get some input from everyone to see if they have strong objections or better suggestions. Thoughts? |
| 2:18 pm [vampwillow] |
TTL on RSS feeds Is there a reason that the LJ code taking in RSS feeds ignores the TTL information? I've noted this because a feed I operate has the following in the User Info: Syndication Status: Last checked: 2004-10-02 06:14:33 Next check: 2004-10-02 07:14:33 ie. it is being checked hourly, yet the TTL set in the RSS datastream is <ttl>240</ttl> ie. should only be being checked every *four* hours. If a feed specifies a TTL shouldn't we actually use that information and not ignore it? (ie. 'be nice'!) |
![[info]](http://web.archive.org/web/20050207063625im_/http://journal.bad.lv/img/userinfo.gif){kind=small}