New ask Hacker News story: Ask HN: How would you spatialize higher dimensional data?
Ask HN: How would you spatialize higher dimensional data?
4 by anigbrowl | 2 comments on Hacker News.
I wrangle a lot of social media datasets, which could be anything from an email database to an old 4chan archive. I'm primarily interested in the metadata network dynamics rather than the content (although I might mine it for hashtags of length/complexity, but I am not very interested in NLP). Sometimes it's just a few thousand items, sometimes orders of magnitude larger. Typically when I represent this data visually I use force-directed layouts or tree/radial ones if it's definitely hierarchical. And I'm pretty good at the visualization side - filtering, community detection, backbone extraction, layer decomposition, motif identification, time windowing and so on. I enjoy starting out with a giant hairball and translating it into something intuitively comprehensible and explicable. I also experiment a lot with other dataviz paradigms - treemaps, hyperbolic geometry and so on. But one thing that bothers me about many layout algorithms is that they're so arbitrary; once you go past a certain level of complexity you can get quite different outcomes from the same dataset and starting procedures. And it's difficult to represent agents and their activity on the same graph, eg influencers and their tweets. I am constantly trying to think of ways to map the raw data to Cartesian space or on the interior of a sphere, not unlike a planetarium. But I have some kind of cognitive or knowledge-based block to understanding this, which I can't seem to get around. How would you spatialize a complex network - eg suppose you had unlimited access to the HN API and plugged it into a particle system or an FPS game engine? I'm more interested in creating a vivid, explorable impression than scientific modeling. I just struggle with the concept of locality because (in this example) any HN user could reply to or vote/flag any other as there's no 'travel time' between threads or users. And without a coherent spatial metaphor, I can't think of good mappings to 2 or 3 dimensions of space and another of time. Thanks for your help!
4 by anigbrowl | 2 comments on Hacker News.
I wrangle a lot of social media datasets, which could be anything from an email database to an old 4chan archive. I'm primarily interested in the metadata network dynamics rather than the content (although I might mine it for hashtags of length/complexity, but I am not very interested in NLP). Sometimes it's just a few thousand items, sometimes orders of magnitude larger. Typically when I represent this data visually I use force-directed layouts or tree/radial ones if it's definitely hierarchical. And I'm pretty good at the visualization side - filtering, community detection, backbone extraction, layer decomposition, motif identification, time windowing and so on. I enjoy starting out with a giant hairball and translating it into something intuitively comprehensible and explicable. I also experiment a lot with other dataviz paradigms - treemaps, hyperbolic geometry and so on. But one thing that bothers me about many layout algorithms is that they're so arbitrary; once you go past a certain level of complexity you can get quite different outcomes from the same dataset and starting procedures. And it's difficult to represent agents and their activity on the same graph, eg influencers and their tweets. I am constantly trying to think of ways to map the raw data to Cartesian space or on the interior of a sphere, not unlike a planetarium. But I have some kind of cognitive or knowledge-based block to understanding this, which I can't seem to get around. How would you spatialize a complex network - eg suppose you had unlimited access to the HN API and plugged it into a particle system or an FPS game engine? I'm more interested in creating a vivid, explorable impression than scientific modeling. I just struggle with the concept of locality because (in this example) any HN user could reply to or vote/flag any other as there's no 'travel time' between threads or users. And without a coherent spatial metaphor, I can't think of good mappings to 2 or 3 dimensions of space and another of time. Thanks for your help!
Comments
Post a Comment